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Preface 


As  a  result  of  continuing  concern  with  large  cost  overruns  in  a  broad  range  of  major 
defense  programs,  Congress  enacted  new  statutory  provisions  extending  the  ambit  of 
the  existing  Nunn-McCurdy  Act,  stipulating  the  review  and  reporting  by  the  Depart¬ 
ment  of  Defense  (DoD)  of  factors  and  issues  in  both  specific  and  general  terms.  In 
accordance  with  the  revised  Nunn-McCurdy  Act,  the  Performance  Assessments  and 
Root  Cause  Analysis  (PARCA)  office  must  provide  its  root  cause  explanation  as  part  of 
a  60-day  program  review  triggered  when  the  applicable  military  department  secretary 
reports  a  breach. 

In  March  2010,  the  newly  created  PARCA  office  within  the  Office  of  the  Sec¬ 
retary  of  Defense  (OSD),  in  view  of  staffing  limitations,  elected  to  rely  on  federally 
funded  research  and  development  center  (FFRDC)  support  to  help  discharge  its  new 
responsibilities.  It  engaged  the  RAND  Corporation  to  study  the  root  causes  of  Nunn- 
McCurdy  breaches  in  four  of  six  major  defense  acquisition  programs  that  had  breached: 
the  DDG-1000,  the  Joint  Strike  Fighter,  the  Longbow  Apache,  and  the  Wideband 
Global  Satellite.  RAND  conducted  its  work  within  this  60-day  window  so  that  the 
PARCA  office  could  use  RAND  findings  in  preparing  DoD  reports  to  Congress. 

This  monograph  contains  the  findings  from  each  of  the  program  reviews  con¬ 
ducted  by  RAND.  Although  the  programs  involved  are  different,  the  RAND  analysis 
found  many  similarities,  ranging  from  unrealistic  expectations  to  contracting  meth¬ 
ods.  The  report  details  the  specifics  of  the  reported  breaches  and  is  intended  for  a  gen¬ 
eral  audience;  however,  it  also  used,  in  part,  previously  conducted  RAND  research  on 
programs  such  as  the  Joint  Strike  Fighter  and  the  DDG-1000.  Selected  publications 
that  may  be  of  interest  about  these  programs  and  related  acquisition  processes  are  avail¬ 
able  at  www.rand.org. 

This  research  was  sponsored  by  OSD  PARCA  and  conducted  within  the  Acquisi¬ 
tion  and  Technology  Policy  Center  of  the  RAND  National  Defense  Research  Insti¬ 
tute,  a  federally  funded  research  and  development  center  sponsored  by  the  Office  of  the 
Secretary  of  Defense,  the  Joint  Staff,  the  Unified  Combatant  Commands,  the  Navy, 
the  Marine  Corps,  the  defense  agencies,  and  the  defense  Intelligence  Community. 
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For  more  information  on  the  RAND  Acquisition  and  Technology  Policy  Center, 
see  http://www.rand.org/nsrd/ndri/centers/atp.html  or  contact  the  director  (contact 
information  is  provided  on  the  web  page). 
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Summary 


Background  and  Purpose 

As  a  result  of  continuing  program  cost  growth  and  observations  by  the  Government 
Accountability  Office  (GAO)  placing  defense  acquisition  on  the  high-risk  target  list, 
Congress  continued  to  refine  and  expand  on  program  execution  controls.  In  this  report, 
we  focus  on  the  mandates  for  root  cause  analysis  and  performance  appraisal  of  major 
defense  acquisition  programs  (MDAPs)  and  how  RAND  supported  DoD  (that  is,  the 
director  of  the  PARCA  office  in  the  Office  of  the  Under  Secretary  of  Defense,  Acqui¬ 
sition,  Technology,  and  Logistics  [OUSD  AT&L])  as  it  executed  its  responsibilities 
for  analysis  and  certification  of  MDAPs  that  had  breached  Nunn-McCurdy  thresh¬ 
olds.  The  research  and  analysis  performed  by  RAND  and  the  conclusions  it  reached 
formed  a  foundation  of  the  inherently  governmental  function  of  program  certification 
required  by  statute. 

In  the  face  of  six  MDAP  cost  threshold  breaches  reported  by  several  secretaries  of 
the  military  departments,  the  PARCA  director  determined  that  he  required  support  to 
execute  his  statutory  responsibilities  and  turned  to  FFRDCs  and  academia  to  provide 
that  support  for  the  program  execution  status  research  and  analysis.  Along  with  the 
Institute  for  Defense  Analyses  (IDA),  Center  for  Naval  Analyses  (CNA),  and  the  Uni¬ 
versity  of  Tennessee,  RAND  was  engaged  to  perform  analysis  and  provide  recommen¬ 
dations.  RAND  was  assigned  sole  responsibility  for  three  programs:  Wideband  Global 
Satellite  (WGS),  Long  Bow  Apache  Helicopter  (Apache  Block  III),  and  Zumwalt-CXass 
Destroyer  (DDG-1000).  In  addition,  RAND  shared  the  analytic  effort  for  a  fourth 
program,  the  Joint  Strike  Fighter  (JSF),  with  IDA  and  the  University  of  Tennessee. 

For  RAND,  these  efforts  were  somewhat  unusual  on  several  levels.  The  time 
allowed  under  the  statute  from  the  moment  the  breach  was  announced  to  the  need 
for  certification  was  very  short.  The  time  limit  drove  the  use  of  already  available  mate¬ 
rial  held  by  the  government  to  perform  analysis.  Research  was  conducted  in  concert 
with  government  offices  (in  point  of  fact,  it  could  not  have  happened  otherwise  given 
the  time  available),  often  under  the  direction  of  DoD  offices  and  officials.  DoD  offi¬ 
cials  used  the  results  of  RAND’s  research  and  analysis  by  performing  their  inherently 
governmental  functions  well  before  the  production  of  even  a  draft  report.  As  a  conse¬ 
quence  of  the  resulting  approach  to  this  shared  set  of  responsibilities,  the  Secretary  of 
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Defense  was  able  to  perform  his  certification  responsibilities  to  allow  program  continu¬ 
ation.  The  materials  found  in  Chapters  Three  through  Six  of  this  report  represent  the 
results  of  the  RAND  research  and  analysis  efforts  used  by  DoD  in  arriving  at  program 
certification. 

As  noted  above,  time  did  not  allow  for  generation  of  fresh  data.  DoD  made  avail¬ 
able  to  four  RAND  research  and  analysis  teams  (one  for  each  of  the  designated  pro¬ 
grams)  the  cost,  schedule,  and  performance  data  in  existing  DoD  databases.  The  four 
RAND  teams,  each  headed  by  a  senior  RAND  researcher,  performed  data  analysis, 
engaged  both  applicable  contractor  and  appropriate  government  personnel  in  in-depth 
discussions  on  program  status,  and  reviewed  pertinent  previous  RAND  efforts  and 
documentation  to  distill  program  understanding. 

Each  program  represented  a  different  set  of  conditions  and,  as  a  result,  required 
tailored  approaches  to  produce  the  requisite  material  in  support  of  the  government’s 
process.  The  fundamental  questions  to  be  answered  were  stipulated  in  statute,  and  the 
PARCA  office  established  an  approach  to  display  the  findings  at  a  summary  level.  Both 
are  discussed  in  greater  detail  in  Chapters  One  and  Two.  Some  of  the  causes  identi¬ 
fied,  such  as  quantity  changes  and  unfounded  technical  expectations,  were  found  in 
most  of  the  programs.  Several  unique  findings  led  to  an  examination  in  greater  depth 
of  the  metrics  available  to  enable  progress  review.  That,  in  turn,  led  to  a  finding  that 
greater  attention  needs  to  be  paid  to  the  metrics  currently  in  vogue  because  they  do 
not  appear  to  cover  adequately  some  aspects  of  program  execution.  This  issue  becomes 
even  more  significant  in  light  of  another  finding.  To  satisfy  statutory  demands  fully, 
the  root  cause  analysis  and  performance  assessment  processes  need  to  be  conducted  on 
a  continuum  that  examines  and  reports  on  the  same  issues  over  time,  thereby  requiring 
more  than  static  metrics.  Finally,  given  the  state  of  the  industrial  base,  a  better  under¬ 
standing  is  required  of  the  interrelationship  of  government  and  commercial  lines  of 
business  within  a  defense  contractor  as  well  as  the  importance  of  the  financial  health 
of  any  contractor  supplying  goods  and  services  to  DoD.1 

What  is  certain  is  that  if  DoD  is  to  continue  to  perform  the  PARCA  office  func¬ 
tions  as  it  did  in  the  initial  process,  a  better  understanding  of  the  full  range  of  data 
available  needs  to  be  developed,  and  access  to  those  data  needs  to  be  better  enabled. 
Of  particular  interest  and  discussed  further  in  Chapter  Seven  is  greater  access  to  the 
Defense  Acquisition  Management  Information  Retrieval  (DAMIR)  system  and  to  ser¬ 
vice  program  briefings. 


1  Loren  Thompson  of  the  Lexington  Institute  emphasizes  that  latter  point  in  his  recent  article  on  the  financial 
travails  of  Oshkosh.  See  Loren  B.  Thompson,  “Oshkosh  Vote  Shows  Danger  of  Rewarding  Aggressive  Bids  from 
Shaky  Companies,”  Lexington  Institute  Early  Warning  Blog,  September  3,  2010. 
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Results 

Root  Causes 

Although  our  work  on  root  causes  identified  several  contributory  factors,  our  analy¬ 
sis  of  the  four  programs  indicates  that  three  were  common  across  all  four  programs: 
planning,  changes  in  the  economy,  and  program  planning.  Table  S.l  provides  a  sum¬ 
mary  of  the  root  cause  analyses  of  the  four  programs  reviewed,  listing  the  causes  in 

Table  S.l 

Comparison  Matrix  of  the  Root  Causes  of  Program  Cost  Growth 


Category 

Root  Cause  of  Nunn-McCurdy  Breach 

WGS 

Apache 

DDG-1000 

JSF 

Planning 

Underestimate  of  baseline  cost 

V 

V 

V 

V 

Ambitious  scheduling  estimates 

V 

V 

Poorly  constructed  contractual  incentives 

vv 

V 

Immature  technologies 

vv 

V 

vv 

Ill-conceived  manufacturing  process 

V 

Unrealistic  performance  expectations 

V 

Delay  in  awarding  contract 

V 

Insufficient  RDT&E 

V 

V 

V 

V 

Changes  in 

Increase  in  component  costs 

H 

V 

V 

V 

economy 

Increase  in  labor  costs 

V 

V 

Discontinued/decreased  production  of  components 

V 

Decreased  demand  for  similar  technology  in  private 
sector  (economies  of  scale) 

V 

Inflation 

V 

V 

V 

V 

Program 

management 

Production  delays 

H 

V 

vv 

Change  in  procurement  quantities 

Increase 

V 

H 

Decrease 

vv 

V 

Unanticipated  design,  manufacturing,  and 
technology  integration  issues 

VV 

V 

vv 

Lack  of  government  oversight  or  poor  performance 
by  contractor  personnel 

V 

V 

Inadequate  or  unstable  program  funding 

V 

V 

V 

V 

Accounting  artifact 

V 

NOTE:  V  =  Root  cause,  V  V  =  Significant  root  cause. 
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each  of  the  three  categories.  A  single  check  mark  indicates  a  root  cause,  and  double 
check  marks  indicate  a  significant  root  cause.2  As  the  table  shows,  underestimation  of 
baseline  cost;  increases  in  component  costs;  insufficient  Research,  Development,  Test, 
and  Evaluation  (RDT&E);  inflation;  and  increased,  inadequate,  or  unstable  program 
funding  were  identified  as  root  causes  in  all  four  programs.  For  the  WGS,  the  increase 
in  component  costs  was  a  significant  root  cause.  The  prevalence  of  these  same  factors 
across  four  very  different  programs  may  indicate  systemic  root  causes  that  warrant 
increased  attention  in  future  program  planning. 

Lessons  Learned 

Our  analysis  of  the  root  causes  of  the  Nunn-McCurdy  breaches  led  us  to  draw  the  fol¬ 
lowing  lessons  learned: 

•  Production  delays  increase  exposure  to  changing  private  sector  market  condi¬ 
tions,  which  can  result  in  cost  growth. 

•  Acquisition  flexibility  (e.g.,  start-stop  programs)  comes  with  a  cost. 

•  Cost  estimates  should  be  conducted  independently  of  a  program  manager. 

•  Combining  remanufactured  and  new  build  items  causes  complexity  and  can  lead 
to  cost  growth. 

•  Greater  planning  of  manufacturing  process  organization  is  required. 

•  Large  reductions  in  procurement  quantities  can  significantly  increase  per  unit 
cost. 

•  Sufficient  RDT&E  is  required  to  ensure  the  “produce-ability”  of  a  program. 

•  Greater  government  oversight  of  the  contractor  is  required  in  a  technologically 
complex  project. 

•  More  “hedges”  against  risky  elements  of  program  are  required. 

•  Additional  collaboration  is  needed  on  design  specifications  and  discussion  of  cost- 
performance  trade-offs. 

These  lessons  can  help  project  managers  avoid  cost  increases  if  they  are  attended 
to  promptly  in  the  early  phases  of  the  program.  For  example,  when  a  program  has 
obvious  technical  complexity,  the  program  manager  should  take  steps  early  in  the 
project  to  ensure  that  the  government  has  made  adequate  provision  for  oversight  of  the 
contractor. 


2  The  discrimination  between  a  root  cause  and  significant  root  case  was  a  qualitative  assessment  of  the  magni¬ 
tude  of  the  effect  in  contributing  to  the  Nunn-McCurdy  breach. 
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Introduction 


Background 

The  U.S.  Congress  has  become  increasingly  concerned  with  the  rate  of  cost  increases 
in  major  defense  acquisition  programs  (MDAPs).  This  concern,  combined  with  the 
prospect  of  shrinking  defense  budgets,  led  Congress  to  focus  more  of  its  senior  policy¬ 
makers’  attention  on  oversight  of  these  programs.1  The  Weapon  Systems  Acquisition 
Reform  Act  (WSARA)  of  20092  established  a  number  of  requirements  that  affected 
the  operation  of  the  Defense  Acquisition  System  and  the  duties  of  the  key  officials  who 
support  it,  including  the  requirement  to  establish  a  new  organization  in  the  Office  of 
the  Secretary  of  Defense  (OSD)  with  the  mandate  to  conduct  and  oversee  Performance 
Assessments  and  Root  Cause  Analysis  (PARCA)  for  MDAPs.3  The  act  assigned  the 
PARCA  organization  five  primary  responsibilities: 

1.  Carry  out  performance  assessments  of  MDAPs. 

2.  Perform  root  cause  analysis  (RCA)  of  MDAPs  whose  cost  growth  exceeds  the 
threshold  as  detailed  in  the  Nunn-McCurdy  Act. 

3.  Issue  policies,  procedures,  and  guidance  governing  the  conduct  of  performance 
assessments  and  root  cause  analyses. 

4.  Evaluate  the  utility  of  performance  metrics  used  to  measure  the  cost,  schedule, 
and  performance  of  MDAPs. 

5.  Advise  acquisition  officials  on  performance  issues  that  may  arise  regarding  an 
MDAP.4 


1  U.S.  House  of  Representatives,  House  Report  111-124  on  S.  454,  “Weapon  Systems  Acquisition  Reform  Act 
of  2009,”  May  20,  2009.  U.S.  House  of  Representatives,  Committee  on  Armed  Services,  House  Report  111-101 
on  H.R.  2101,  “Weapons  Acquisition  System  Reform  Through  Enhancing  Technical  Knowledge  and  Oversight 
Act  of  2009,”  May  12,  2009. 

2  Public  Law  111-23,  Weapon  Systems  Acquisition  Reform  Act  of  2009,  May  22,  2009. 

3  Ashton  B.  Carter,  Under  Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics,  OUSD  (AT&L), 
Directive-Type  Memorandum  (DTM)  09-027 — Implementation  of  the  Weapon  Systems  Acquisition  Reform 
Act  of  2009,  December  4,  2009;  Public  Law  111-23,  Section  103. 

4  Public  Law  111-23,  Section  103. 
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Project  Objective 

PARCA’s  responsibilities  are  inherently  governmental  functions.  However,  recognizing 
the  limited  number  of  staff  at  its  disposal,  the  PARCA  office  reached  out  to  federally 
funded  research  and  development  centers  (FFRDCs)  to  assist  it  with  the  analytic  por¬ 
tions  of  these  inherently  governmental  functions. 

This  report  focuses  on  a  key  aspect — program  research  and  analysis — of  one  of 
these  responsibilities:  the  RCA  process.  10  USC  [U.S.  Code]  §  2433a  stipulates  that 
a  root  cause  analysis  be  conducted  by  the  director  of  the  PARCA  office  when  a  major 
weapon  system  acquisition  program  incurs  a  critical  cost  growth  breach,  as  detailed  in 
the  Nunn-McCurdy  Act.5  The  WSARA  requires  that  the  Secretary  of  Defense  conduct 
an  RCA  after  consultation  with  the  Joint  Requirements  Oversight  Council  (JROC).6 
The  RCA  assesses  the  underlying  cause  or  causes  of  shortcomings  in  cost,  schedule, 
or  performance  that  may  have  contributed  to  the  MDAP’s  critical  cost  overrun.  The 
WSARA  defined  topics  to  consider  as  part  of  an  RCA,  including  the  role  played,  if 
any,  of  the  following: 

1.  unrealistic  performance  expectations 

2.  unrealistic  baseline  estimates  for  the  cost  or  schedule 

3.  immature  technologies  or  excessive  manufacturing  or  integration  risk 

4.  unanticipated  design,  engineering,  manufacturing,  or  technology  integration 
issues  arising  during  program  performance 

5.  changes  in  procurement  quantities 

6.  inadequate  program  funding  or  funding  instability 

7.  poor  performance  by  government  or  contractor  personnel  responsible  for  pro¬ 
gram  management 

8.  any  other  matters.7 

At  the  time  this  report  was  written,  six  ongoing  major  weapon  systems  programs 
had  incurred  critical  Nunn-McCurdy  cost  growth  breaches  in  2010  and  thus  were 
subject  to  the  provisions  of  current  federal  law  described  above.  RAND  was  enlisted 
by  the  organization  responsible  for  PARCA  to  support  the  RCA  process  for  four  of 
these  MDAPs:  the  DDG-1000  Zum  wait- Class  Destroyer,  the  Joint  Strike  Fighter  (JSF 
or  F-35),  Longbow  Apache  Block  III  (AB3),  and  Wideband  Global  Satellite  (WGS).8 
Given  the  time  constraints  mandated  by  the  Nunn-McCurdy  recertification  process, 


5  The  Nunn-McCurdy  provision  and  definition  of  critical  cost  growth  are  described  in  greater  detail  in  Chapter 
Two. 

®  10  USC  §  2433a,  as  specified  in  Public  Law  111-23,  Section  206. 

7  Public  Law  111-23,  Section  103;  Carter,  2009. 

8  The  RCA  for  the  Joint  Strike  Fighter  was  performed  in  coordination  with  the  Institute  for  Defense  Analyses 
and  the  University  of  Tennessee. 
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RAND  organized  four  teams  to  provide  a  “quick  look”  assessment  of  the  major  issues 
related  to  cost,  schedule,  or  performance  shortcomings. 

Table  1.1  summarizes  the  cost  growth  breaches  incurred  by  these  programs.  Chap¬ 
ter  Two  and  the  related  Appendix  A  provide  detail  on  the  characteristics  of  breaches. 
Chapters  Three  to  Six  discuss  the  specific  breaches  of  each  program  in  detail. 


Approach 

Each  team  undertook  two  tasks  in  tandem:  establishing  the  basic  facts  surrounding 
the  program’s  Nunn-McCurdy  breach  and  determining  the  contribution  to  unit  cost 
growth  from  the  eight  RCA  issues  stipulated  in  the  WSARA  legislation  and  listed 
above.  The  time  that  elapsed  between  each  program’s  completion  of  Milestone  B 
(MS  B)  and  the  recognition  of  RCA  issues  applicable  to  each  program  was  portrayed 
in  a  chart  similar  to  that  of  Table  1.2,  which  illustrates  the  framework  provided  by 
the  PARCA  office.  For  each  program  under  RAND’s  purview,  this  figure  provided  a 
temporal  lens  through  which  RCA  issues  could  be  viewed  and  the  analysis  informed. 
In  addition  to  the  Endings  for  each  program,  a  WGS-,  AB3-,  DDG-1000-,  and  JSF- 
specific  version  of  this  chart  appears  in  subsequent  chapters  of  this  report. 

The  table  arrays  the  issues  specified  in  the  Nunn-McCurdy  legislation  in  the  left 
column  and  the  fiscal  years  of  a  notional  program  whose  MS  B  occurred  in  2001  across 
the  top.  X  indicates  the  years  in  which  the  event  occurred.  Thus,  unrealistic  cost  or 
schedule  estimates  occurred  every  year  after  MS  B.  The  text  under  execution  for  2002 
is  in  place  of  an  X,  as  the  change  represented  in  the  text  is  large  and  warrants  further 
explanation  in  the  chart. 

As  mentioned  above,  the  time  line  for  reporting  and  certification  provided  in  the 
statute  restricted  the  amount  of  data  gathering  and  analysis  that  could  be  performed. 
To  meet  PARCA  needs,  RAND  relied  on  many  documents  through  the  course  of  the 
project.  Not  all  were  used  through  the  course  of  the  project,  but  the  following  list  gives 
a  good  idea  of  the  documentation  that  was  available  for  the  RAND  team  during  the 
short  window  of  work: 

From  Defense  Acquisition  Management  Information  Retrieval: 

•  SARs 

•  Defense  Acquisition  Executive  Summary  (DAES)  reports 

•  Program  Objective  Memoranda  (POMs) 

•  Budget  Estimate  Submissions  (BESs) 

•  APBs 

•  earned  value  management  (EVM)  contract  data. 


Table  1.1 

Current  Programs  Incurring  Cost  Growth  Breaches  in  2010  (FY  $  Millions) 


Current  Cost  Growth  Threshold  Breaches 

Estimate  - 


Program 

Baseline  Unit 

Cost 

December 

2009  SAR 

Baseline 

Breached 

Percentage 

Amount 

Level 

Baseline 

Quantity 

December  2009 
SAR  Quantity 

Cause  in  SAR 

SAR 

Explanation 

WGS 

FY  2001 

APUC 
$294,160 
2007  APB 

APUC 

$374,186 

Over  current 
baseline 
(2007  APB) 

APUC 

+27.20 

+$80,026 

FY  2001 

Critical 

5 

7 

Quantity  change 
from  5  satellites 
to  7  satellites 

Addition 
of  two  new 
satellites  (WGS 

7  and  8) 

PAUC 
$358,520 
2007  APB 

PAUC 

$424,457 

PAUC 

+18.39 

+$65,937 

FY  2001 

Significant 

5 

7 

APUC 
$268,200 
2000  APB 

APUC 

$374,186 

Over  original 
baseline 
(2000  APB) 

APUC 

+39.52 

+$105,986 

FY  2001 

Significant 

3 

7 

AB3 

FY  2006 

APUC 

$9,600 

2007  APB 

PAUC 

$11,139 

2007  APB 

APUC 

$12,591 

PAUC 

$13,977 

Over  current 
baseline 
(2007  APB) 

APUC 

+31.16 

PAUC 

+25.48 

+$2,991 

FY  2006 

+$2,838 

FY  2006 

Critical 

Critical 

639 

634 

695 

690 

Quantity  change 
from 

remanufacture 
of  634  aircraft 
to  building 
additional 

56  new  aircraft 

56  new  build 
aircraft 
significantly 
more  expensive 
to  build 
because  they 
are  100%  new 
versus  30% 
new  for  a 
remanufacture 

APUC 

$9,225 

2006  APB 

APUC 

$12,591 

Over  original 
baseline 
(2006  APB) 

APUC 

+36.49 

+$3,366 

FY  2006 

Significant 

597 

690 

DDG-1000 

FY  2005 

APUC 
$2,323,470 
2005  APB 

APUC 

$2,901,967 

Over  current 
baseline 
(2005  APB) 

APUC 

+24.90 

+$578,497 

FY  2005 

Significant 

10 

3 

Quantity  change 
from  10  ships  to 

3  ships 

Truncation 
of  program 
from  10  ships 
in  November 
2005  APB  to  3 
ships  in  PB11 

PAUC 
$3,154,790 
2005  APB 

PAUC 

$5,882,500 

PAUC 

+86.46 

+$2,727,710 
FY  2005 

Critical 

10 

3 

PAUC 

$3,154,790 

PAUC 

$5,882,500 

Over  original 
baseline 

PAUC 

+86.46 

+$2,727,710 
FY  2005 

Critical 

10 

3 

2005  APB  (2005  APB) 
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Table  1.1 — Continued 


Current 

Estimate 

December 

2009  SAR 

Cost  Growth  Threshold  Breaches 

Program 

Baseline  Unit 

Cost 

Baseline 

Breached 

Percentage 

Amount 

Level 

Baseline 

Quantity 

December  2009 
SAR  Quantity 

Cause  in  SAR 

SAR 

Explanation 

JSF  (F-35) 
FY  2002 

APUC  $50,245 
2001  APB 

APUC 

$79,003 

Over  original 
baseline 
(2001  APB) 

APUC 

+57.24 

+$28,758 
FY  2002 

Critical 

2,852 

2,443 

Increases  in 
procurement  cost 
estimates 

Historical 
increases  and 
programmatic 
changes 
reported  in 
previous  SARs 
and  estimated 
procurement 
cost  increases 

PAUC  $61,793 
2001  APB 

PAUC 

$97,110 

PAUC 

+57.15 

+$35,317 
FY  2002 

Critical 

2,866 

2,457 

Increases  in 
procurement  cost 
estimates 

Historical 
increases  and 
programmatic 
changes 
reported  in 
previous  SARs 
and  estimated 
procurement 
cost  increases 

SOURCES:  Department  of  Defense,  Selected  Acquisition  Report  WGS,  December  31,  2009;  Department  of  Defense,  Selected  Acquisition  Report  AB3, 
December  31,  2009;  Department  of  Defense,  Selected  Acquisition  Report  DDG-1000,  December  31,  2009;  Department  of  Defense,  Selected  Acquisition 
Report  F-35,  December  31,  2009. 


NOTES:  The  numbers  in  red  indicate  the  "speeding  ticket"  triggering  root  cause  analysis  by  PARCA.  Formulas  and  criteria  used  to  generate  the 
information  shown  here  are  specified  in  federal  law  and  are  shown  in  Chapter  Two.  APUC  =  average  procurement  unit  cost;  PAUC  =  program 
acquisition  unit  cost;  APB  =  acquisition  program  baseline;  SAR  =  Selected  Acquisition  Report;  PB  =  President's  Budget. 
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Table  1.2 

PARCA  Root  Cause  Matrix  Framework 


Year  from  MS  B  and  Fiscal  Year 

B 

+1 

+2 

+3 

+4 

+5 

2001 

2002 

2003 

2004 

2005 

2006 

Baseline  issues 

Unrealistic  estimates  for 
cost  or  schedule 

X 

X 

X 

X 

X 

Immature  technology; 
excessive  manufacturing, 
integration  risk 

X 

X 

X 

X 

X 

Unrealistic  performance 
expectations 

X 

X 

X 

X 

X 

Execution  issues 

Changes  in  procurement 

X 

Change  from 

quantity 

150  to  55 

Inadequate  funding/ 
funding  instability 

X 

Unanticipated 
design,  engineering, 
manufacturing,  or 
technical  issues 

X 

Poor  performance  of 
government  or  contract 
personnel 

X 

From  Cost  Analysis  Improvement  Group/Cost  Assessment  and  Program  Eval¬ 
uation  (CAIG/CAPE)  documents  on  costs  throughout  the  life  of  the  program  and 
Nunn-McCurdy  breach: 

•  Latest  acquisition  strategy 

•  Congressional  Budget  Office  (CBO)  cost  estimates 

•  The  draft  and  final  Cost  Analysis  Requirements  Description  (CARD) 

•  Acquisition  Decision  Memoranda  (ADMs)  post-Milestone  B 

Government  and  industry  collaboration  website: 

•  award  fee 

•  cost  estimates 

•  Contractor  Performance  Assessment  Reporting  System  (CPARS) 

•  design  reviews 
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•  prime  contractor  contracts 

•  program  documents. 

Contractor  monthly  assessment  reports. 

Briefings  from: 

•  prime  contractor 

•  Program  Office  and  Joint  Program  Office  (JPO)  on  the  System  Development  and 

Demonstration  (SDD),  System  Requirements  Review  (SRR),  and  Unit  Recurring 

Flyaway  (URF) 

•  Overarching  Integrated  Process  Team  (OIPT)  on  Nunn-McCurdy  breach 

•  deputy  advisors  working  group 

•  Naval  Air  Systems  Command  (NAVAIR) 

•  Naval  Sea  Systems  Command  (NAVSEA) 

•  Program  Office  post-Milestone  B. 

Meetings  with: 

•  prime  contractors 

•  Program  Office 

•  JPO 

•  CAPE 

•  Air  Force  Cost  Analysis  Agency  (AFCAA). 

Internal  DoD  memos  from  John  Young,  Aston  Carter,  Acquisition  Executives. 

“Notification  to  Congress”  of  the  Nunn-McCurdy  breach. 

Some  of  this  material  and  the  program  discussion  process  provided  quantitative 
data  and  some  provided  program  execution  and  management  decisionmaking  insights. 

Analysis  of  each  MDAP  identified  risk  as  a  common  denominator  to  all  four  pro¬ 
grams.  The  teams  investigated  a  wide  variety  of  risk,  ranging  from  the  rather  straight¬ 
forward,  such  as  insufficiency  of  program  funding,  to  the  more  complex,  such  as  reli¬ 
ance  on  the  private  sector  to  maintain  the  optimal  technology  configuration.  As  the 
risks  encountered  by  each  program  were  identified  and  assessed,  sources  of  program 
vulnerabilities  were  collected  and  compared.  The  compilation  of  these  vulnerabilities 
represents  a  key  contribution  of  this  work:  the  “vulnerability  matrix.”  The  vulner¬ 
abilities  that  were  identified  throughout  the  research,  in  addition  to  the  time  period 
or  phase  at  which  this  vulnerability  could  potentially  occur,  are  listed  in  Table  1.3.  A 
more  developed  version  of  this  table,  populated  in  accordance  with  the  findings  of  each 
team,  appears  in  Chapter  Seven.  Several  programs  shared  some  vulnerabilities,  whereas 
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Table  1.3 

Program  Vulnerabilities  That  May  Contribute  to  Cost  Growth 


Type  of  Event 

Time  Period/Phase 

Cost  Events 

Cost  increases  began 

Concept  refinement 

Cost  increases  began 

Technology  development 

Cost  increases  began 

System  development  and  demonstration 

Cost  increases  began 

Production  and  deployment 

Cost  increases  began 

Operations  and  support  (O&S) 

Cost  increases  began 

Additional  blocks 

Documented  different  estimates 

Concept  refinement 

Documented  different  estimates 

Technology  development 

Documented  different  estimates 

System  development  and  demonstration 

Documented  different  estimates 

Production  and  deployment 

Documented  different  estimates 

O&S 

Documented  different  estimates 

Additional  blocks 

Known  Factors  During  Cost  Estimation 

Unrefined  design  and  performance  requirements 

All 

Lack  of  previous  similar  work 

All 

No  or  bad  past  performance 

All 

Technology  immaturity 

All 

Technical  complexity 

All 

Complexity  of  system  integration 

All 

Decision  to  employ  parallel  development  rather 
than  sequential 

All 

Insufficient  number  of  government  personnel  to 
run  the  program 

All 

Unique  contracting  strategy 

All 

Ambitious  system  performance  goals 

All 

Difficulties  in  negotiating  major  contracts  (e.g., 
contract  protests) 

All 

Internal  Changes  to  Program 

Change  of  performance  requirements 

Concept  refinement 

Change  of  performance  requirements 

Technology  development 

Change  of  performance  requirements 

System  development  and  demonstration 

Change  of  performance  requirements 

Production  and  deployment 

Change  of  performance  requirements 

Operations  and  support 
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Table  1.3 — Continued 


Type  of  Event 

Time  Period/Phase 

Cost  Events 

Funding  instability 

All 

Change  in  quantity — increase 

All 

Change  in  quantity — decrease 

All 

Changes  in  program  guidance/service  mission 

All 

External  Changes  to  Program  Environment 

Increase/decrease  in  quantity  of  related  program 

All 

Cancellation  of  predecessor  program 

All 

Changes  to  external  programs 

All 

Addition  of  new  construction  to  an  upgrade 
program 

All 

Change  of  technology  direction 

System  development  and  demonstration 

Changes  to  Operating  Environment 

Increase  in  wage  rates 

All 

Increase  in  cost  of  materials  or  equipment 

All 

Changes  to  industrial  base 

All 

Change  in  government  standards  or  regulations 

All 

Natural  disaster/act  of  God 

All 

Significant  congressional  interest 

All 

others  were  specific  to  only  one.  It  is  important  to  note  that  this  matrix  is  not  defini¬ 
tive  or  all-inclusive;  rather,  it  is  a  product  that  will  evolve  over  time,  as  a  perspective 
through  which  MDAPs  can  be  monitored  and  evaluated. 


Organization  of  This  Report 

This  report  contains  seven  chapters.  The  next  chapter  summarizes  how  Congress  has 
used  legislation  to  establish  and  increase  its  oversight  of  weapon  system  acquisition 
programs  and  discusses  the  requirements  of  Nunn-McCurdy  legislation  in  more  detail. 
Those  familiar  with  the  history  of  congressional  oversight  of  defense  acquisition  pro¬ 
grams  and  the  provisions  of  Nunn-McCurdy  may  wish  to  skip  that  chapter.  Chapters 
Three  through  Six  report  the  findings  of  the  RCAs  performed  by  the  RAND  teams  on 
each  of  the  four  programs  under  review  in  the  following  sequence:  DDG-1000  Zumwalt- 
Class  Destroyer,  Joint  Strike  Fighter,  Longbow  Apache  ITelicopter,  and  Wideband 
Global  Satellite.  These  chapters,  idiosyncratic  of  the  specific  programs  and  independent 
analyses  of  the  teams,  reflect  the  reports  that  were  sent  to  the  PARCA  office  to  be  used 
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to  carry  out  its  responsibilities  and  produce  the  materials  necessary  for  the  recertifica¬ 
tion  decision  process  as  required  by  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  (USD  AT&L),  and,  ultimately,  the  Secretary  of  Defense. 
The  final  chapter  presents  our  conclusions  and  recommendations. 


CHAPTER  TWO 


Congressional  Concern  with  Growth  in  Program  Costs 


This  chapter  provides  a  synopsis  of  the  history  of  congressional  oversight  of  cost  growth 
and  oversight  of  program  status.  It  begins  with  a  discussion  of  how  current  federal  law 
deals  with  cost  growth,  starting  from  early  SAR  reporting  demands.  It  then  traces 
the  history  of  the  legislation  focused  on  containing  the  cost  growth  of  major  weapon 
systems. 


Selected  Acquisition  Reports 

Starting  in  the  1970s,  when  Congress  expropriated  the  SARs  to  use  for  determining 
program  status,  Congress  has  continued  to  create  mechanisms  to  gain  insights  into 
program  execution.  The  SARs  were  initially  designed  as  an  internal  DoD  program 
status  tool.  DoD  Instruction  7000.3  established  SARs  on  February  23,  1968,  with  the 
following  purpose: 

The  SARs’  initial  purpose  was  to  keep  its  sponsor,  the  Assistant  Secretary  of 
Defense  (Comptroller),  apprised  of  the  progress  of  selected  acquisitions  and  to 
compare  this  progress  with  planned  technical,  schedule,  and  cost  performance. 

In  February  1969,  the  Chairman  of  the  Senate  Armed  Services  Committee  asked 
the  Secretary  of  Defense  to  provide  status  reports  on  major  weapons  systems.  The 
parties  agreed  in  April  1969  that  the  SAR  would  be  the  vehicle  to  satisfy  the  com¬ 
mittee’s  needs.1 

However,  SARs  did  not  become  a  legal  reporting  requirement  until  1975,  with 
Public  Law  94-106. 2  SARs  summarize  the  latest  estimates  of  cost,  schedule,  and  perfor¬ 
mance  status.  These  reports  are  prepared  annually  in  conjunction  with  the  President’s 


1  U.S.  General  Accounting  Office,  “Comptroller  General’s  Report  to  the  Congress:  ‘SARs’ — Defense  Depart¬ 
ment  Reports  That  Should  Provide  More  Information  to  the  Congress,”  May  9,  1980,  p.  1. 

2  Bob  Leach,  “Acquisition  Reporting  Overview,”  Office  of  the  Under  Secretary  of  Defense  (Acquisition,  Tech¬ 
nology,  and  Logistics),  June  12-13,  2002,  pp.  9—11. 
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Budget  and  quarterly  under  certain  conditions.  Congress  has  continued  to  increase  its 
oversight  of  weapon  system  acquisition  programs  that  incur  cost  overruns. 


History  of  Cost  Growth  Legislation 

In  1981,  Senator  Samuel  Nunn  and  Congressman  David  McCurdy  introduced  the 
Nunn-McCurdy  amendment3  to  the  Department  of  Defense  Authorization  Act  of 
1982. 4  The  purpose  of  the  amendment  was  to  establish  congressional  oversight  of 
defense  weapon  system  acquisition  programs  that  experience  cost  growth  above  limits 
specified  in  the  amendment.  The  Nunn-McCurdy  amendment  defined  two  types  of 
unit  cost:  total  PAUC,  which  is  the  sum  of  development  funding  and  procurement 
funding  divided  by  units  procured,  and  APUC,  which  is  the  procurement  funding 
divided  by  units  procured.  Cost  growth  of  a  weapon  system  was  measured  by  how 
much  the  unit  costs  in  1982  exceeded  the  same  respective  unit  costs  in  the  weapon  sys¬ 
tem’s  SAR  dated  March  31,  1981.  Hence,  the  amendment  applied  only  to  those  major 
weapon  systems  with  March  31,  1981,  SARs.  Greater  detail  and  definition  of  terms  is 
provided  in  Appendix  A. 

The  original  amendment  required  that  the  Secretary  of  Defense  notify  Con¬ 
gress  when  a  major  weapon  system  unit  cost  growth  exceeded  15  percent.  If  unit  cost 
growth  exceeded  25  percent,  the  program  was  assumed  terminated  unless  the  Secre¬ 
tary  of  Defense  submitted  specific  written  certifications  to  Congress  within  60  days  of 
making  the  cost  growth  determination.  These  certifications  survive  in  current  law  and 
are  explained  in  detail  below. 

Congress  made  the  provisions  of  the  Nunn-McCurdy  amendment  permanent  in 
the  1983  Authorization  Act5  by  requiring  that  the  secretary  of  each  military  depart¬ 
ment  establish  a  baseline  description  of  each  major  weapon  system  acquisition  program 
under  the  jurisdiction  of  the  secretary.  The  baseline  description  was  to  include  a  base¬ 
line  estimate  of  the  program  cost.  The  permanent  Nunn-McCurdy  provisions  mea¬ 
sured  unit  cost  growth  by  comparing  the  current  unit  costs  against  the  same  respective 
unit  costs  in  the  baseline  estimate.  The  cost  thresholds  for  notifying  Congress  and 
for  program  termination  presumptions  in  the  original  Nunn-McCurdy  amendment 
remained  unchanged  in  the  1983  Authorization  Act  but  have  subsequently  changed. 
The  certifications  required  to  be  submitted  to  Congress  also  remain  unchanged  in  cur¬ 
rent  federal  law. 


3  The  Nunn-McCurdy  amendment  is  also  known  as  the  Nunn-McCurdy  provision.  See  Nunn-McCurdy 
Amendment,  Department  of  Defense  Authorization  Act,  1982,  Report  No.  97-311,  November  3,  1981. 

4  Public  Law  97-86,  National  Defense  Authorization  Act  of  1982,  December  29,  1981. 

5  Public  Law  97-252,  National  Defense  Authorization  Act  of  1983,  September  8,  1982. 
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Congress's  Continuing  Concern  with  Program  Cost  Growth 

Every  two  years  since  1990,  Congress  has  tasked  the  Government  Accountability 
Office  (GAO)  with  creating  a  list  of  issues  considered  to  represent  high  risk.  The  issues 
on  the  high-risk  list  are  those  that  require  attention  because  they  either  are  particu¬ 
larly  vulnerable  to  mismanagement,  waste,  fraud,  or  abuse;  or  need  transformation  to 
address  major  economy,  efficiency,  or  effectiveness  challenges.  The  last  high-risk  list 
came  out  in  January  2009,  and  Department  of  Defense  weapon  systems  acquisition  is 
on  that  list.6  One  primary  reason  weapon  systems  acquisition  is  on  the  high-risk  list 
is  because  of  large  cost  overruns.  To  be  sure,  cost  overruns  are  hardly  new  to  weapon 
systems  acquisition.  Indeed,  weapon  systems  acquisition  has  appeared  on  every  GAO 
high-risk  list  produced  since  1990,  and  each  time,  large  cost  overruns  have  been  a  pri¬ 
mary  reason  for  its  placement  on  the  list. 

The  Department  of  Defense  does  not  necessarily  disagree  with  GAO’s  assessment. 
Shortly  after  the  GAO  issued  its  last  high-risk  list,  Secretary  of  Defense  Robert  Gates 
testified  before  the  House  Armed  Services  Committee  stating  that  chief  among  the 
department’s  concerns  is  acquisition.7  Secretary  Gates  noted  that  acquisition  problems 
have  been  persistent  and  difficult  to  resolve  despite  Congress’s  long  involvement  in 
trying  to  resolve  them  and  the  nearly  130  studies  on  various  aspects  of  acquisition  that 
also  have  not  resulted  in  a  comprehensive,  effective,  and  permanent  solution.  He  stated 
that  the  defense  acquisition  system  cannot  be  reformed  in  a  short  period  of  time  and 
that  cost  overrun  problems  have  plagued  defense  acquisition  since  the  first  Secretary  of 
War,  Henry  Knox,  incurred  cost  overruns  in  building  the  nation’s  Navy. 

Beginning  with  the  passage  of  the  Nunn-McCurdy  amendment  in  1981,  Con¬ 
gress  has  repeatedly  refined  and  increased  its  oversight  of  major  defense  weapon  systems 
acquisition  programs  that  incur  cost  growth  breaches.  When  statutory  provisions  deal¬ 
ing  with  cost  growth  were  first  enacted,  program  personnel  were  very  concerned  about 
breaching  cost  growth  thresholds.  As  the  concept  of  incurring  cost  growth  threshold 
breaches  became  more  familiar,  such  breaches  have  caused  less  apprehension.  Con¬ 
gress’s  latest  attempt  to  increase  oversight  imposed  more  stringent  actions,  including  a 
mandatory  root  cause  analysis  and  program  reassessment  for  programs  incurring  criti¬ 
cal  cost  growth  breaches.  Whether  Congress’s  latest  legislative  efforts  to  control  cost 
growth  prove  to  be  effective  remains  to  be  determined. 

What  is  clear,  starting  with  efforts  in  the  1980s,  is  that  Congress  has  been  dis¬ 
satisfied,  if  not  alarmed,  by  DoD’s  acquisition  performance,  because  defense  acquisi¬ 
tion  constitutes  a  significant  portion  of  discretionary  spending  for  the  United  States. 
Secretary  Gates’s  comments  notwithstanding,  Congress  is  not  prepared  to  wait  for 


6  U.S.  Government  Accountability  Office,  High-Risk  Series:  An  Update,  GAO-09-271,  Washington,  D.C.,  Janu¬ 
ary  2009. 

7  Robert  M.  Gates,  Secretary  of  Defense  statement  submitted  to  the  House  Armed  Services  Committee,  Janu¬ 
ary  27,  2009. 
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acquisition  performance  to  get  better.  The  cost  overrun  experienced  in  DoD  programs 
is  particularly  pernicious  in  the  eyes  of  a  Congress  dealing  with  high  unemployment 
levels  where  military  projects  represent  well-paying  jobs  in  home  districts.  The  Nunn- 
McCurdy  provisions  have  been  Congress’s  way  of  doing  due  diligence  on  the  one  hand 
while  forcing  the  Secretary  of  Defense  to  make  tough  decisions  on  programs  central  to 
the  national  security  on  the  other. 

As  with  all  permanent  legislation,  the  permanent  provisions  of  the  Nunn-McCurdy 
amendment  were  codified  in  the  U.S.  Code.  The  next  section  discusses  the  permanent 
provisions  as  they  currently  exist  and  notes  changes  that  have  occurred  since  the  origi¬ 
nal  Nunn-McCurdy  amendment  was  enacted  into  law  in  the  1982  Department  of 
Defense  Authorization  Act.8 


Program  Cost  Growth  and  Certification  Requirements 

A  weapon  system  acquisition  program  incurs  a  unit  cost  growth  breach  when  the  unit 
cost  reaches  or  exceeds  limits  or  thresholds  specified  in  federal  law.  Such  cost  growth 
breaches  are  commonly  known  as  Nunn-McCurdy  breaches,  in  reference  to  the  origi¬ 
nal  Nunn-McCurdy  amendment,  though  this  term  does  not  appear  in  federal  law. 

Unit  cost  growth  breaches  are  computed  using  definitions,  formulas,  and  thresh¬ 
olds  provided  in  federal  law.  10  USC  §§  2432  and  2433  provide  the  definitions  of  “pro¬ 
gram  acquisition  unit  cost”  and  “procurement  unit  cost,”  formulas  for  computing  unit 
cost  growth,  and  thresholds  for  incurring  significant  and  critical  cost  growth  breaches. 
The  concept  of  significant  cost  growth  and  critical  cost  growth  were  introduced  with 
enactment  of  the  Defense  Authorization  Act  of  2006. 9 

A  significant  cost  growth  threshold  breach  is  defined  as 

•  a  PAUC  that  is  at  least  15  percent  over  the  program  acquisition  unit  cost  shown 
in  the  current  baseline  estimate  for  the  program  or 

•  an  APUC  that  is  at  least  15  percent  over  the  acquisition  procurement  unit  cost 
shown  in  the  current  baseline  estimate  for  the  program  or 

•  a  PAUC  that  is  at  least  30  percent  over  the  program  acquisition  unit  cost  shown 
in  the  original  baseline  estimate  or 

•  an  APUC  that  is  at  least  30  percent  over  the  acquisition  procurement  unit  cost 
shown  in  the  original  baseline  estimate. 


8  The  original  Nunn-McCurdy  amendment  was  not  considered  permanent  law,  because  it  applied  only  to 
March  31,  1981,  SARs.  Nonpermanent  laws,  such  as  the  yearly  congressional  authorization  amounts,  are  not 
typically  codified  in  the  U.S.  Code.  Permanent  provisions  are  always  codified. 

9  Public  Law  109-163,  Department  of  Defense  Authorization  Act  of  2006,  Section  802,  January  6,  2006. 
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A  critical  cost  growth  threshold  breach  is  defined  as 

•  a  PAUC  that  is  at  least  25  percent  over  the  program  acquisition  unit  cost  shown 
in  the  current  baseline  estimate  for  the  program  or 

•  an  APUC  that  is  at  least  25  percent  over  the  acquisition  procurement  unit  cost 
shown  in  the  current  baseline  estimate  for  the  program  or 

•  a  PAUC  that  is  at  least  50  percent  over  the  program  acquisition  unit  cost  shown 
in  the  original  baseline  estimate  or 

•  an  APUC  that  is  at  least  50  percent  over  the  acquisition  procurement  unit  cost 
shown  in  the  original  baseline  estimate. 

Table  2.1  summarizes  the  unit  cost  threshold  definitions  in  federal  law. 

In  addition,  the  Weapon  Systems  Acquisition  Reform  Act  of  200910  imposes  a 
root  cause  analysis  process  on  programs  that  incur  critical  breaches.  This  action  is 
required  of  the  Secretary  of  Defense  after  consultation  with  the  Joint  Requirements 
Oversight  Council.* 11  The  pertinent  legal  definitions  appear  in  Appendix  A. 

Current  law  specifies  that  a  program  that  incurs  a  critical  cost  growth  breach  is 
assumed  to  be  terminated  unless  the  Secretary  of  Defense  provides  specific  certifica¬ 
tions  to  Congress  within  60  days  of  the  SAR  due  date  as  specified  in  10  USC  §  2432(f). 


Table  2.1 

Breach  Thresholds 


Level 

Unit  Cost 

Baseline 

Threshold,  % 

Source 

Significant 

PAUC 

Current 

>15 

10  USC  §  2433  (a)(4)(A)(i) 

APUC 

Current 

>15 

10  USC  §  2433  (a)(4)(B)(i) 

PAUC 

Original 

>30 

10  USC  §  2433  (a)(4)(A)(ii) 

APUC 

Original 

>30 

10  USC  §  2433  (a)(4)(B)(ii) 

Critical 

PAUC 

Current 

>25 

10  USC  §  2433  (a)(5)(A)(i) 

APUC 

Current 

>25 

10  USC  §  2433  (a)(5)(B)(i) 

PAUC 

Original 

>50 

10  USC  §  2433  (a)(5)(A)(ii) 

APUC 

Original 

>50 

10  USC  §  2433  (a)(5)(B)(ii) 

10  Public  Law  111-23. 

11  10  USC  §  2433a,  as  specified  in  Section  206  of  Public  Law  111-23. 
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The  certifications  must  state  the  following: 

•  The  system  is  essential  to  the  national  security. 

•  No  alternatives  will  provide  an  acceptable  capability  to  meet  joint  military  require¬ 
ments  at  less  cost;  the  new  estimates  of  PAUC  and  APUC  have  been  determined 
by  the  director  of  Cost  Analysis  and  Program  Evaluation  to  be  reasonable. 

•  The  program  has  a  higher  priority  than  programs  whose  funding  must  be  reduced 
to  accommodate  the  growth  in  cost  of  the  program. 

•  The  management  structure  for  the  program  is  adequate  to  manage  and  control 
the  PAUC  or  APUC. 

These  certifications  must  be  accompanied  by  a  root  cause  analysis  and  assessment. 

10  USC  §  2433a  stipulates  that  a  root  cause  analysis  be  conducted  by  the  director 
of  the  PARCA  office  when  a  major  weapon  system  acquisition  program  incurs  a  critical 
cost  growth  breach.  The  root  cause  analysis  assesses  the  underlying  cause  of  the  criti¬ 
cal  cost  overrun.  The  office  is  responsible  for  producing  a  root  cause  report  that  must 
identify  the  role,  if  any,  of  the  eight  elements  listed  in  Chapter  One  in  causing  the  cost 
overrun. 


Evolvement  of  Statutory  Governance  of  Cost  Growth  Breaches 

History  indicates  that  if  cost  overruns  are  not  controlled  by  Congress’s  latest  efforts  in 
the  WSARA  of  2009,  further  legislative  actions  may  be  in  the  offing.  Indeed,  Congress 
has  stipulated  that  a  new  acquisition  process  be  put  in  place  for  acquiring  information 
technology  to  address  the  problems  unique  to  such  acquisitions.  The  National  Defense 
Authorization  Act  of  2010  also  mandates  an  assessment  by  the  director  of  Cost  Assess¬ 
ment  and  Program  Evaluation  on  the  wisdom  of  creating  baselines  for  operation  and 
support  costs.  In  addition,  GAO  has  been  tasked  by  the  same  act  to  evaluate  growth  in 
these  costs.  These  congressional  requests  indicate  that  Congress  is  considering  legisla¬ 
tive  actions  to  control  operations  and  support  costs  in  addition  to  its  actions  to  control 
procurement  costs.  These  actions  also  indicate  that  Congress  is  determined  to  use  its 
full  authority  to  launch  a  multipronged  attack  to  control  defense  acquisition  cost  over¬ 
runs.  The  Weapons  Acquisition  Reform  Act  of  2009  passed  with  nearly  unanimous 
votes  in  both  houses  of  Congress.  This  strong  support  and  the  wide  spectrum  of  recent 
congressional  actions  with  respect  to  weapon  systems  acquisition  taken  together  indi¬ 
cate  that  Congress  is  determined  to  continue  the  quest  initially  proposed  by  the  origi¬ 
nal  Nunn-McCurdy  amendment  to  control  cost  growth  overruns. 

The  next  four  chapters  address  the  four  programs  assigned  to  RAND  for  analy¬ 
sis;  similar  information  is  available  for  the  two  programs  assigned  to  the  Institute  for 
Defense  Analyses  (IDA).  As  noted  above,  the  analysis  responsibility  for  JSF  was  shared. 
Although  decisions  shared  by  DoD  and  the  Congress  are  notable  in  the  “speeding  tick- 
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ets”  involved  (quantity  changes  are  pervasive,  for  example,  in  most  of  these  programs), 
Congress  looks  to  DoD  to  control  costs.  It  is  notable  that  all  the  programs  received 
recertification. 


CHAPTER  THREE 


Zumwalt- Class  Destroyer  (DDG-1000) 


On  February  1,  2010,  the  Navy  notified  OSD  and  Congress  of  a  Nunn-McCurdy 
breach  in  the  DDG-1000  program.  The  immediate  cause  of  the  breach  was  a  reduction 
in  the  planned  total  number  of  ships  from  ten  to  three,  which  resulted  in  an  86  per¬ 
cent  increase  in  PAUC  and  a  25  percent  increase  in  APUC.1  This  chapter  describes  the 
DDG-1000  program,  the  Nunn-McCurdy  breach,  and  the  research  approach  taken  to 
analyze  the  breach  (including  factors  that  limit  that  analysis). 


Program  Overview 

The  DDG-1000  program  has  its  origins  in  a  1993  study  of  candidate  system  concepts 
for  a  surface  combatant  for  the  21st  century  (SC  21).  The  DDG-1000  represents  an 
entirely  new  ship  system  designed  around  a  highly  specialized  mission:  precision  strike 
and  volume  fire  support  from  the  littoral  against  land  targets.  The  system  has  both 
offensive  and  defensive  capabilities  designed  to  contribute  surface,  subsurface,  and  air 
battlespace  dominance  in  the  littoral. 

The  ship  hull  is  a  wave-piercing  tumblehome  configuration — a  new  design  for  a 
ship  of  this  size  (approximately  15,000  ton  displacement).  The  integrated  power  system 
(IPS),  also  new  on  a  Navy  ship,  consists  of  two  main  turbine  generators,  two  auxiliary 
turbine  generators,  and  two  advanced  induction  motors  (AIMs).  The  deckhouse  is 
made  of  composite  material  rather  than  the  traditional  steel. 

Mission  systems  include  80  advanced  vertical  launch  cells  placed  around  the 
periphery  of  the  ship  (as  opposed  to  the  center  as  on  current  ships)  containing  a  mix  of 
Tomahawk,  evolved  Sea  Sparrow  missiles  (ESSMs),  standard  missiles,  and  the  advanced 
gun  system  (AGS)  155-millimeter  guns  forward  of  the  deckhouse  with  600  rounds  of 
the  long  range  land  attack  projectile  (LRLAP)  developed  specifically  for  this  mission. 
There  are  two  57-millimeter  close-in  gun  systems  (CIGS)  for  self-defense.  The  aviation 


1  PAUC  is  calculated  as  the  total  Research,  Development,  Test,  and  Evaluation  (RDT&E)  and  procurement 
(construction)  costs  divided  by  the  planned  total  quantity.  APUC  is  total  procurement  cost  divided  by  total 
quantity. 
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contingent  is  designed  to  include  two  MH-60R  helicopters  or  one  MH-60R  and  three 
vertical  takeoff  and  landing  unmanned  aerial  vehicles  (VTUAVs).  Two  seven-meter 
rigid  hull  inflatable  boats  (RHIBs)  or  two  11-meter  RHIBs  can  be  accommodated.  The 
dual  band  radar  (DBR)  includes  an  S-band  volume  search  radar  (VSR)  and  X-band 
multifunction  radar  (MFR)  for  enhanced  operations  in  the  cluttered  littoral  environ¬ 
ment.  The  high-resolution  multifrequency  (HR/MF)  bow  sonar  arrays  and  a  multi¬ 
function  towed  array  provide  subsurface  operational  capabilities. 


PAUC  Breach 

The  DDG-1000  program  has  only  one  development  estimate  and  that  is  the  APB 
approved  on  November  23,  2005.2  This  November  2005  APB,  approved  at  Mile¬ 
stone  B,  is  both  the  current  baseline  estimate  and  the  original  baseline  estimate  used  to 
compute  the  PAUC  and  APUC  in  the  DDG-1000  SARs. 

The  DDG-1000  December  2009  SAR  shows  three  cost  growth  breaches  for 
the  DDG-1000  program.3  Table  3.1  shows  a  significant  APUC  breach  of  more  than 
25  percent  and  a  critical  PAUC  breach  of  86  percent  of  the  baseline.  Since  the  original 
and  current  baselines  for  DDG-1000  are  the  same  (November  2005  APB),  according 
to  the  unit  cost  growth  criteria,  the  APUC  growth  is  a  significant  breach  against  the 
current  baseline,  and  the  PAUC  growth  is  a  critical  breach  against  both  the  original 
and  current  baselines.  The  baseline  quantity  of  ten  ships  was  used  to  compute  the  base¬ 
line  APUC  and  PAUC;  a  quantity  of  three  ships  was  used  to  compute  the  December 
2009  current  estimate  APUC  and  PAUC. 

The  two  critical  PAUC  cost  growth  breaches  are  the  “speeding  tickets”  that  trig¬ 
gered  the  root  cause  analysis  specified  in  the  WSARA  Act  of  2009.  The  December 


Table  3.1 

DDG-1000  Unit  Cost  Growth  Breaches  (FY  2005  $  Millions) 


Baseline 

(November  2005) 

Current  Estimate 
(December  2009) 

Difference 

Cost 

Quantity 

Cost 

Quantity 

Cost 

Percentage 

APUC 

2,323.5 

10 

2,902.0 

3 

578.5 

24.9 

PAUC 

3,154.8 

10 

5,882.5 

3 

2,727.7 

86.46 

SOURCE:  Department  of  Defense,  Selected  Acquisition  Report  DDG-1000,  December  31,  2009. 


2  Department  of  Defense,  Acquisition  Program  Baseline  (APB)  DDG  1000,  November  23,  2005- 

3  A  schedule  breach  associated  with  an  eight-day  delay  award  of  the  detail  design  and  construction  contract  is 
also  documented  in  the  December  2009  SAR.  This  breach  is  nominally  unrelated  to  the  cost  growth  and  is  not 
addressed  here. 
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2009  SAR  states  that  the  quantity  change  from  ten  ships  to  three  ships  was  the  cause 
for  breaches  and  further  explains  that  truncation  of  the  DDG-1000  program  from  ten 
ships  in  the  November  2005  APB  to  three  ships  in  the  2011  President’s  Budget  caused 
the  breaches. 

Figure  3.1  shows  the  change  in  PAUC,  APUC,  and  quantity  since  the  original 
program  conceptual  baseline  in  January  1998.  The  November  2005  baseline  was  estab¬ 
lished  at  Milestone  B  and  is  the  baseline  from  which  the  cost  increases  are  calculated. 
The  original  program  envisioned  a  32-ship  class,  which  dropped  to  eight  in  the  middle 
of  the  technology  development  phase  (Phase  III  in  program  terminology)  as  the  ship 
design  and  technology  was  matured  and  as  more  realistic  cost  estimates  were  gener¬ 
ated.  A  baseline  quantity  of  ten  ships  was  approved  at  Milestone  B.  Planned  quantity 
was  reduced  to  seven  as  part  of  the  FY  2008  budget  preparation;  a  revised  APB  was 
generated  by  the  Program  Office,  approved  up  through  the  Navy  Acquisition  Execu¬ 
tive,  and  sent  to  OUSD  AT&L  for  approval.  That  revised  baseline — with  a  quantity 
of  seven  ships — was  never  formally  approved  by  OUSD  AT&L  because  of  questions  at 
that  time  concerning  the  final  quantity.  Figure  3.1  also  shows  how  sensitive  PAUC  is 
to  changes  in  quantity. 


Figure  3.1 

Unit  Cost  and  Quantity  Changes  Since  Program  Inception 
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Research  Approach  and  Caveats 

The  DDG-1000  is  a  complex  program  with  a  long  history  rich  in  details,  unique 
attributes,  and  technical  and  programmatic  accomplishments.  This  discussion  is  not 
intended  to  be  a  complete  program  history  and  so  does  not  attempt  to  deal  with  every 
element  of  the  program.  Rather,  we  have  attempted  to  identify  aspects  of  the  program 
that  are  relevant  to  the  explanation  of  the  Nunn-McCurdy  unit  cost  breaches.  For  the 
most  part,  the  focus  is  on  the  baseline  established  at  Milestone  B  and  on  subsequent 
program  execution. 

The  work  documented  here  was  done  within  the  construct  of  the  response  to  a 
critical  Nunn-McCurdy  breach,  which  requires  either  recertification  or  termination 
of  the  program  within  60  days  of  informing  Congress  of  the  breach.  The  information 
contained  here  has  been  validated  to  the  extent  possible  and  draws  heavily  on  offi¬ 
cial  primary  source  documentation.  Historical  and  recent  documentation  include  the 
official  ADMs,  DAES,  SARs,  OSD  CAIG  memos,  budget  documents,  and  Program 
Office  briefings  dating  from  Milestone  B  to  the  present.  We  also  draw  on  briefings 
that  the  Program  Office  gave  to  the  Nunn-McCurdy  Integrated  Product  Teams  (IPTs) 
established  to  generate  the  information  required  for  the  determination  of  recertifica¬ 
tion  or  termination.  Discussions  with  Program  Office  and  OSD  officials  supplemented 
the  primary  source  documentation. 

We  also  draw  heavily  on  prior  RAND  research,  particularly  on  a  case  study  spon¬ 
sored  by  the  future  destroyer  program  (DD(X))/DDG  1000  Program  Office  that  was 
conducted  over  the  period  March  2004  to  December  2007.  A  final  report  describing 
the  program  in  detail  from  the  earliest  studies  through  Milestone  B  awaits  public 
release  by  the  sponsor.  A  follow-on  report  covering  program  history  through  award  of 
the  dual  lead  ship  construction  contracts  in  February  2008  is  ongoing.  These  separate 
research  activities  collected  a  considerable  amount  of  program  information  (reports, 
briefings,  oversight  documentation,  and  notes  from  interviews  with  program  officials), 
all  of  which  was  consulted  in  support  of  the  root  cause  analysis  when  relevant. 


Findings4 

The  DDG-1000  December  2009  SAR  reported  a  critical  PAUC  Nunn-McCurdy 
breach  (86.46  percent)  due  to  a  PB11  budget  decision  to  truncate  the  program  at  three 
ships.  The  critical  breach  consequently  required  a  root  cause  analysis  by  PARCA.  The 
PAUC  grew  from  a  Milestone  B  baseline  objective  of  $3,154.8  million  to  a  current  esti¬ 
mate  of  $5,882.5  million,  largely  as  a  result  of  spreading  RDT&E  costs  across  fewer 
ships. 


4 


All  dollar  figures  in  this  narrative  are  expressed  in  fiscal  year  2005  dollars. 
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Our  analysis  confirms  the  Navy’s  explanation  that  the  quantity  change  from  ten 
to  three  was  the  main  driver  of  the  unit  cost  growth.  The  86.46  percent  increase  in  the 
PAUC  is  largely  due  to  spreading  total  program  RDT&E  costs,  which  are  not  sensi¬ 
tive  to  changes  in  procurement  quantity,  over  three  ships  rather  than  ten,  as  shown 
in  Table  3.2.  Of  the  $2,727.71  million  difference  between  the  November  2005  APB 
PAUC  and  the  December  2009  PAUC,  nearly  80  percent  is  from  redistributing  the 
RDT&E,  and  just  over  20  percent  of  it  is  from  increases  in  the  Ship  Construction 
Navy  (SCN)  account.  Nonquantity-related  cost  growth  (which,  as  of  December  2009 
was  less  than  8  percent  over  the  November  2005  APB  RDT&E  figure)  did  not  cause 
the  Nunn-McCurdy  breach.  Table  3.2  shows  the  breakdown  of  differences  between 
the  SAR  baseline  APB  of  November  2005  and  the  December  2009  SAR. 

To  broaden  our  perspective  on  the  effect  of  truncating  DDG-1000  ship  quanti¬ 
ties,  we  created  a  cost  estimate  for  a  hypothetical  “three-ship  program  at  Milestone  B,” 
using  baseline  data  from  the  2005  SAR.  Table  3.3  shows  the  result.  Interestingly, 
the  calculated  PAUC  estimate  is  higher  than  the  current  estimate  that  put  the  pro¬ 
gram  in  breach.  RDT&E  costs  grew  from  the  baseline  amount  of  $8,313.2  million  to 
$8,941.6  million  in  the  current  estimate  (both  in  FY  2005  $)  largely  driven  by  the  addi¬ 
tional  scope  of  the  development  effort,  including  additional  funding  for  LRLAP  low 
rate  production  to  support  round  qualification,  self-defense  test  ship  support,  next  gen¬ 
eration  command  and  control  processor,  enterprise  alternative  to  full  ship  shock  trials, 
and  line  of  sight/below  line  of  sight  installation.  RDT&E  growth  is  largely  explained 
by  the  addition  of  subsystems  to  the  scope  of  work,  rather  than  to  any  technical  prob¬ 
lems  encountered  during  development.  In  contrast,  the  procurement  account  (SCN) 
funding  we  estimated  for  a  nominal  three-ship  program  at  Milestone  B  is  higher  than 
the  current  estimate  SCN.  Though  only  rough  estimates,  these  results  suggest  that  if 
adjusted  for  changes  in  quantity,  DDG-1000  would  not  have  experienced  any  growth 
in  unit  cost  through  the  December  2009  current  estimate.  Unit  cost  growth  is  largely 

Table  3.2 

Breakdown  of  Differences  Between  SAR  Baseline  APB  (November  2005)  PAUC  and  Current 
PAUC  Estimate  in  PB11  (FY  2005  $  Millions) 


Current  Difference 

SAR  Baseline  Estimate  - 


Category 

2005  APB 

Quantity 

(PB11) 

Quantity 

Cost 

Percentage 

RDT&E 

831.32 

10 

2,980.53 

3 

2,149.21 

78.79 

Procurement  flyaway 
recurring 

2,172.67 

10 

2,374.07 

3 

201.40 

7.38 

Procurement  flyaway 
nonrecurring 

150.80 

10 

527.90 

3 

377.10 

13.82 

Total  (PAUC) 

3,154.79 

10 

5,882.50 

3 

2,727.71 

86.46 

SOURCE:  Department  of  Defense,  Selected  Acquisition  Report  DDG-1000,  December  31,  2009. 
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Table  3.3 

"Three-Ship  Program"  Cost  Comparisons  (FY  2005  $  Millions) 


APB  MS  B 
(Current  APB 
Objective) 

Estimate  APB 
MS  B 

Current  Estimate 
(New  APB) 

RDT&E 

8,313.2 

8,313.2 

8,941.6 

SCN 

23,234.7 

9,799.6 

8,705.9 

Total 

31,547.9 

18,112.8 

17,647.5 

Quantity 

10 

3 

3 

PAUC 

3,154.79 

6,037.6 

5,882.5 

SOURCES:  Department  of  Defense,  Selected  Acquisition  Report  DDG-1000, 

December  31,  2005;  Department  of  Defense,  Selected  Acquisition  Report 
DDG-1000,  December  31,  2009. 

NOTES:  The  "three-ship  program"  cost  estimate  is  calculated  by  summing 
advanced  procurement,  nonrecurring  costs  through  the  first  three  ships, 
and  the  budget  lines  for  the  first  three  ships,  as  shown  in  Table  15  of 
the  December  2005  SAR.  If  the  ramp-down  in  SCN  costs  in  years  after 
last  ship  is  funded  ($412.6  million)  is  included,  total  estimated  PAUC  = 

$6,175.13  million. 

driven  by  a  relatively  small  increase  in  RDT&E  funds  (7.5  percent)  to  cover  additional 
work  and  the  spreading  of  total  program  RDT&E  funds  over  fewer  ships  (three  ships 
rather  than  ten). 

The  statute  that  established  PARCA  provided  a  list  of  potential  root  causes  for 
cost  breaches.  PARCA  turned  these  into  a  matrix.  The  PARCA  root  cause  matrix  for 
DDG-1000  is  shown  in  Table  3.4;  subsequent  paragraphs  elaborate  on  each  set  of  fac¬ 
tors.  It  is  useful  to  divide  the  discussion  of  program  history  and  key  events  into  two 
categories:  establishing  the  baseline  and  program  execution. 

Baseline  Issues 

Unrealistic  Baseline  Estimates  for  Cost  or  Schedule.  The  DDG-1000  cost  esti¬ 
mates  at  Milestone  B  were  lower  than  the  estimates  produced  by  the  CAIG,  CBO, 
and  GAO.  They  were  also  lower  than  historical  experience  would  suggest,  particularly 
when  compared  with  the  DDG-51.  The  differences  in  estimates  related  to  the  level  of 
RDT&E  required  to  mature  critical  technologies  (i.e.,  the  Engineering  Design  Models 
[EDMs])  and  to  integration  of  those  technologies  during  ship  construction,  as  well  to 
the  costs  of  procuring  some  subsystems,  software  development,  and  cost  improvement 
curve  effects  (assumptions  about  first  unit  cost  and  the  slope  of  the  curve).  Though  the 
estimates  in  the  Milestone  B  baseline  were  much  more  realistic  than  earlier  estimates, 
they  were  still  highly  optimistic  given  the  large  uncertainty  surrounding  both  techni¬ 
cal  and  programmatic  aspects  of  the  program. 
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Table  3.4 

PARCA  Root  Cause  Narrative  Matrix  for  DDG-1000 


Year  from  MS  B  and  Fiscal  Year 

B 

1 

2 

3  4 

5 

2005 

2006 

2007 

2008  2009 

2010 

Baseline  issues 

Unrealistic  estimates  for  cost  or  schedule 

X 

X 

Immature  technology;  excessive 
manufacturing,  integration  risk 

X 

Unrealistic  performance  expectations 

X 

Execution  issues 

Changes  in  procurement  quantity 

Change  from 

Change 

10  to  7 

from  7  to  3 

Inadequate  funding/funding  instability 

X 

X 

X  X 

X 

Unanticipated  design,  engineering, 
manufacturing,  or  technical  issues 

X 

X 

X 

Poor  performance  of  government  or  contract 
personnel 

X 

Other 

X 

Table  3.5  shows  the  differences  between  the  Program  Office  and  the  CAIG  cost 
estimates  at  Milestone  B,  both  for  the  initial  plan  proposed  by  the  Navy  and  for  the 
dual  lead  ship  strategy  eventually  adopted. 

The  schedule  estimates  were  perhaps  more  realistic,  but  achieving  them  required 
actually  receiving  the  planned  resources.  Several  program  restructures  pushed  back 
program  milestones  before  the  schedule  at  Milestone  B  was  adopted.  The  Milestone  B 
baseline  schedule  was  based  more  on  the  need  to  maintain  workload  and  financial  sta¬ 
bility  at  the  two  remaining  U.S.  shipyards5  on  estimates  of  when  the  program  would 
be  ready  to  proceed  to  subsequent  technical  engineering  and  contractual  events.  The 
detail  design  and  construction  (DD&C)  contract  was  negotiated  for  nine  months 
(November  2005  to  August  2006)  before  it  was  awarded.  Negotiations  for  the  lead 
ship  construction  award  began  almost  immediately  after  and  required  several  Defense 
Acquisition  Board  internal  program  reviews  (DAB  IPRs);  the  contract  was  eventually 
awarded  in  February  2008,  after  a  delay  of  14  months  caused  by  a  Navy  decision  to 
revisit  contract  type  for  the  lead  ships.  The  extended  contract  negotiations  resulted 
in  pushing  back  out-year  milestones  associated  with  the  DD&C  awards — first  ship 


5  Bath  Iron  Works,  a  subsidiary  of  General  Dynamics,  and  Northrop  Grumman  Ship  Systems,  a  division  of 
Northrop  Grumman. 
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Table  3.5 

CAIG  and  Navy  Cost  Estimates  at  MS  B  ($  Billions) 


PLCCE 

CAIG 

Initial 

Updated 

Initial 

Updated 

Sunk 

3.7 

3.7 

3.7 

3.7 

RDT&E 

4.7 

4.9 

5.3 

5.6 

Procurement 

24.4 

24.4 

30.1 

30.0 

O&S 

17.2 

17.2 

19.5 

19.5 

Total 

49.9 

50.1 

58.7 

58.7 

SOURCES:  OSD  CAIG  Mmemorandum  to  OSD  (AT&L),  Update  to  the 
OSD  CAIG  Report  for  Milestone  B  Defense  Acquisition  Board  (DAB)  of 
the  DD(X)  Program,  November  2,  2005,  April  2005  and  November  2005 
(updated). 

NOTE:  All  estimates  are  in  FY  2005  base  year  dollars  for  a  ten-ship 
program. 

deliveries,  operational  evaluation  (OPEVAL),  initial  operational  capability  (IOC),  and 
Milestone  C,  all  of  which  were  breached. 

Immature  Technologies  or  Excessive  Manufacturing  or  Integration  Risk.  The 

EDMs  did  much  to  mature  key  technologies  and  provided  experience  in  both  design 
and  manufacture  that  increased  confidence  in  achieving  desired  capabilities.  However, 
the  technology  readiness  scores  of  about  half  the  EDMs  were  technology  readiness 
level  5  (TRL  5)  at  Milestone  B,  and  more  than  half  were  expected  to  be  TRL  6  at  ship 
installation.  Thus,  at  Milestone  B,  significant  risk  remained  in  some  major  subsystems, 
including  IPS,  DBR,  and  AGS/LRLAP. 

The  manufacturing  process  envisioned  at  the  time  of  Milestone  B  included  having 
both  shipyards  participate  in  detail  design  for  different  zones  in  the  ship  construction 
plan  as  well  as  each  yard  having  the  ability  to  construct  the  ship.  Each  shipyard  was 
also  responsible  for  the  construction  of  specific  zones,  which  then  would  be  shipped  to 
the  other  yard  for  installation  as  required.  This  plan  was  intended  to  only  maintain  the 
viability  of  both  yards.  There  was  little  historical  precedent  for  managing  this  plan.  The 
deckhouse  was  to  be  designed  and  constructed  out  of  composites,  which  would  entail 
both  new  manufacturing  and  materials  handling  processes,  and  it  was  to  be  manufac¬ 
turing  at  Northrop  Grumman  Ship  Systems  (NGSS)  for  all  ships  in  the  class. 

The  CAIG  noted  that  system  integration  was  underestimated  in  the  Navy  cost 
estimates  at  Milestone  B.  A  Navy  Program  Manager’s  Advisory  Group  (PMAG)  reached 
the  same  conclusion  in  2007.  The  Total  Ship  Computing  Environment  (TSCE)  and 
the  mission  system  equipment  (MSE)  were  consistently  identified  as  significant  inte¬ 
gration  challenges  before  and  after  Milestone  B.  TSCE  required  a  degree  of  integration 
of  both  ship  and  mission  systems  unprecedented  in  Navy  ship  programs.  The  relatively 
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high  level  of  automated  functions  on  DDG-1000  (including  firefighting,  engine  con¬ 
trol,  and  gun-loading)  relied  on  successfully  integrating  those  functions  within  TCSE 
to  realize  the  reduced  manpower  objective;  reduced  manning  was  critical  to  achieving 
reduced  lifecycle  costs. 

Unrealistic  Performance  Expectations.  The  capabilities  envisioned  for  DDG- 
1000  were  very  ambitious  and  required  technologies  that  had  not  been  previously  dem¬ 
onstrated.  The  initial  requirements  for  DD-21,  the  program  from  which  the  DD-1000 
evolved,  derived  from  several  early  studies  and,  together  with  a  change  in  Navy  focus 
from  blue  water  to  the  littoral,  required  many  new  technologies  and  concepts  that  had 
never  been  applied  to  a  large  surface  combatant.  Not  only  did  those  technologies  need 
to  work  individually,  they  also  needed  to  be  highly  integrated  to  improve  or  enhance 
functionality  and  reduce  potential  conflict.  There  was  only  limited  historical  experi¬ 
ence  on  which  to  judge  technical  feasibility,  let  alone  cost  or  schedule.  As  a  result,  there 
was  little  evidence  supporting  the  feasibility  of  achieving  the  desired  performance,  par¬ 
ticularly  under  the  cost  and  schedule  constraints  that  were  later  imposed. 

Execution  Issues 

Changes  in  Procurement  Quantities.  Changes  in  quantity  are  never  the  primary 
source  of  a  change  in  cost.  Rather,  quantity  changes  are  always  driven  by  some  other 
factor,  such  as  a  change  in  threat  or  mission,  which  changes  the  requirement,  or  techni¬ 
cal  problems,  which  increase  costs  and  therefore  affect  affordability. 

The  initial  reductions  in  planned  quantities  from  the  32-ship  class  originally  envi¬ 
sioned  for  DD-21  to  the  ten  ships  included  in  the  Milestone  B  baseline  were  due  to 
affordability.  As  the  system  design  matured  and  experience  was  gained  with  the  key 
technologies  and  subsystems  through  the  EDMs,  more  realistic  (higher)  cost  estimates 
were  developed,  which  reduced  both  the  production  rate  (number  of  ships  approved  for 
construction  in  a  given  year)  and  total  quantity. 

The  baseline  approved  at  Milestone  B  contained  ten  ships,  but  it  was  recognized 
at  the  time  that  this  was  a  planning  number  rather  than  a  hard  requirement  based  on 
mission  needs  analysis.  There  were  significant  cost  risks  still  at  Milestone  B,  and  the 
program  was  potentially  underfunded  (i.e.,  Navy  compared  with  CAIG  funding  esti¬ 
mates).  Those  risks  translated  to  an  affordability  concern,  especially  when  considered 
in  light  of  the  Navy’s  force  structure  goal  of  313  ships.  Post  MS  B,  planned  total  quan¬ 
tity  changed  from  ten  to  seven  and  then  from  seven  to  three.  The  APB  containing  the 
planned  seven-ship  program  was  never  formally  approved  by  OUSD  (AT&L). 

The  quantity  change  from  ten  to  three  that  resulted  in  the  critical  Nunn-McCurdy 
breach  to  PAUC  and  the  significant  APB  breach  to  APUC  were  due  to  perceived 
changes  in  the  emerging  threat  and  mission  priorities.  Navy  leadership  argued  in  July 
2008  that  the  ballistic  missile  defense  (BMD)  mission  had  become  higher  priority  than 
volume  fire  support  from  the  littoral,  and  that  the  DDG-51  hull  was  more  suitable  for 
that  BMD  mission.  The  Navy  actually  proposed  truncating  the  program  at  two  ships 
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(the  two  lead  ships,  one  at  each  yard).  A  subsequent  FY  2010  budget  decision  by  the 
Secretary  of  Defense  increased  the  number  of  ships  to  three  and  moved  construction 
of  all  ships  to  Bath  Ironworks  while  restarting  the  DDG-51  line  at  NGSS.  A  revised 
baseline  in  2008  reflecting  a  quantity  of  seven  was  not  approved  by  OUSD  (AT&L) 
because  of  uncertainty  in  the  total  program  quantity. 

The  FY  2011  budget  decision  to  truncate  the  DDG-1000  program  at  three  ships 
and  restart  DDG-51  production  was  largely  due  to  a  change  in  the  perceived  threat 
and  mission  priority  by  Navy  senior  leadership.  Priority  was  placed  on  ballistic  missile 
defense  rather  than  the  original  DDG-1000  precision  and  volume  fire  support  mis¬ 
sion.  The  radar  hull  study  recommended  the  DDG-51  hull  form  with  a  new  advanced 
missile  defense  radar  (AMDR)  as  more  effective  in  the  ballistic  missile  defense  mission 
than  DDG-1000.  DDG-51  with  AMDR  was  also  assessed  to  have  less  cost  risk.6 

Inadequate  Program  Funding  or  Funding  Instability.  The  Navy  estimated  a 
$600  million  shortfall  in  FY  2006—2011  budgets  at  the  time  of  Milestone  B.7  The 
CAIG  estimated  a  much  higher  shortfall:  $4.1  billion  over  the  FY  2007-2011  budget 
period.8  The  difference  was  largely  due  to  the  differences  in  cost  estimates.  Flowever, 
the  fact  that  the  Navy  had  a  budget  shortfall  using  its  cost  estimates  for  planning  sug¬ 
gests  inadequate  program  funding  at  the  time  of  Milestone  B. 

OUSD  (AT&L)  instructed  the  Navy  at  MS  B  to  fund  to  the  Navy  estimate,  and 
it  did  in  FY  2007.  However,  in  subsequent  years,  the  Navy  did  not  fully  fund  to  the 
program  manager’s  estimate.  This  forced  the  program  to  either  reduce  or  redistribute 
costs  (and  the  associated  work)  across  future  years;  contract  negotiations  and  capability 
(e.g.,  the  number  of  vertical  launch  tubes  went  from  128  to  80,  the  number  of  LRLAPs 
per  gun  went  from  600  to  300)  yielded  some  cost  savings.9 

Figure  3.2  shows  that  RDT&E  funds  have  largely  remained  unchanged  in  the 
program.  Projections  of  out-year  budgets  have  been  relatively  stable  both  before  and 
after  Milestone  B.  The  RDT&E  growth  that  resulted  from  the  increased  scope  dis¬ 
cussed  above  can  be  seen  in  the  difference  between  the  projections  in  the  December 
2005  SAR  and  subsequent  SARs  in  the  period  2009-2013,  but,  relatively  speaking, 
the  difference  is  small.  A  stable  RDT&E  budget  line  facilitates  effective  execution  of 
the  development  program  and  also  indicates  that  few  technical  problems  have  been 
encountered  that  could  not  be  handled  within  existing  budget  authority. 

Figure  3.3  tells  a  very  different  story  for  SCN  (procurement)  funds.  Each  out-year 
projection  of  SCN  funding  has  been  different  from  that  approved  at  Milestone  B,  and 


6  The  DDG-1000  root  cause  analysis  team  has  not  reviewed  the  radar  hull  study  and  so  cannot  validate  its 
results  and  implications. 

7  OSD  CAIG  Memorandum  to  USD  (AT&L),  Update  to  the  OSD  CAIG  Report  for  Milestone  B  Defense  Acquisi¬ 
tion  Board  (DAB)  of  the  DD(X)  Program ,  November  2,  2005,  p.  3. 

8  OSD  CAIG  Memorandum  to  USD  (AT&L),  2005. 

9  We  have  not  yet  been  able  to  verify  the  specific  budget  changes  in  each  year. 
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Figure  3.2 

DDG-1000  RDT&E  Funding 


SOURCES:  Selected  Acquisition  Reports,  December  2005,  December  2006,  December  2007,  and  December 
2009. 
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Figure  3.3 

DDG-1000  Procurement  (SCN)  Funding 


SOURCES:  Selected  Acquisition  Reports,  December  2005,  December  2006,  December  2007,  and  December 
2009. 
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each  has  differed  from  the  other.  With  the  exception  of  the  dramatic  profile  change 
in  the  2009  projection  that  resulted  from  the  quantity  reduction  from  ten  to  three 
ships,  the  other  differences  in  budget  projections  have  been  due  to  factors  other  than 
quantity.  Some  evidence  suggests  that  budget  instability  was  driven  largely  by  Navy  (as 
opposed  to  OSD  or  Congress)  actions  and  that  this  instability  adversely  affected  pro¬ 
gram  execution,  but  we  have  not  yet  been  able  to  verify  this  hypothesis  independently. 

Anecdotal  evidence  also  suggests  that  the  extended  DD&C  contract  negotiations 
(for  both  detail  design  and  lead  ship  construction)  were  due  in  part  to  disagreements 
on  cost,  with  the  Navy  pushing  the  contractors  to  lower  the  price  to  accommodate  the 
Navy’s  budget  targets.  Again,  we  have  been  unable  to  verify  this  claim. 

Unanticipated  Design,  Engineering,  Manufacturing,  or  Technology  Integration 
Issues  Arising  During  Program  Performance.  Some  manufacturing  techniques  were 
new  as  was  the  planned  relationship  between  shipyards,  as  noted  above.  In  addition, 
materials  were  new,  particularly  the  composite  deckhouse,  requiring  completely  new 
construction  procedures.  However,  with  only  20  percent  of  the  lead  ship  completed  to 
date,10  many  of  these  factors  have  not  yet  come  into  play. 

Testing  of  the  EDMs  before  Milestone  B  caused  some  required  redesign  work  to 
improve  performance,  but  there  is  no  evidence  that  this  adversely  affected  program 
cost  or  schedule.  In  fact,  the  purpose  of  the  EDMs  was  to  reduce  risk  earlier  in  the 
program,  thus  reducing  the  cost  of  any  redesign  work. 

It  is  still  too  early  in  the  lead  ship  construction  process  to  determine  whether  the 
significant  integration  risks  mentioned  above  will  result  in  extended  schedules  or  cost 
growth.  However,  those  risks  are  real  and  remain  a  serious  management  and  technical 
challenge. 

That  said,  it  is  also  important  to  note  that  the  detail  design  for  DDG-1000  was 
more  mature  when  construction  began  than  most  other  ship  programs  have  histori¬ 
cally  seen.  This  factor  might  mitigate  some  of  the  manufacturing  and  integration  chal¬ 
lenges  facing  the  program. 

Poor  Performance  by  Government  or  Contractor  Personnel  Responsible  for  Pro¬ 
gram  Management.  The  program  greatly  benefited  from  the  continuity  of  talented 
staff,  both  military  and  civilian,  most  of  whom  had  extended  tenure  at  the  program. 
However,  the  official  Program  Office  was  smaller  than  other  ship  Program  Offices, 
numbering  fewer  than  20  government  officials,  who  drew  on  Systems  Engineering 
and  Technical  Assistance  SETA)  support  locally  as  well  as  on  an  extended  network  of 
support  from  other  NAVSEA  organizations.  The  small  number  of  government  staff 
resulted  in  heavy  workloads  given  the  complexity  of  the  program.  There  is  no  direct 
evidence  that  this  affected  program  execution  or  PAUC  or  APUC  estimates,  but  it 
seems  like  a  noteworthy  and  potentially  contributing  factor.  The  2007  PMAG  noted 
that  the  small  staff  size  was  one  of  its  top  three  factors  affecting  program  execution. 


10 


This  figure  is  a  rough  estimate  provided  during  Program  Office  conversations  with  the  analysis  team  in  2007. 
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Other  Factors  Affecting  Program  Execution.  The  Secretary  of  the  Navy’s  direc¬ 
tion  to  reassess  contracting  strategy  delayed  the  lead  ship  contract  award  for  14  months. 
According  to  the  Program  Office,  this  delay  resulted  in  an  increase  of  approximately 
$500  million  to  the  lead  ship;  over  that  14-month  delay,  the  shipyards  revised  and 
increased  many  cost  elements  in  their  proposed  contract  value. 


Acquisition  Strategy  and  Execution-Related  Observations 

Although  it  may  not  be  possible  to  tie  any  elements  of  the  DDG-1000  acquisition 
strategy  to  the  root  cause  of  the  Nunn-McCurdy  breach,  these  elements  did  play  a  sig¬ 
nificant  role  in  program  execution,  both  positive  and  negative.  The  acquisition  strategy 
and  program  history  have  many  unique  elements  relative  to  past  ship  acquisition  pro¬ 
grams,  including  the  following: 

•  The  relative  newness  of  the  mission  need  and  resulting  requirements  on  ship 
design,  driven  largely  by  the  shift  of  emphasis  from  blue  water  operations  as  part 
of  a  battle  group  to  more  independent  operations  in  the  littoral  in  support  of 
ground  troops.  This  change  reflects  in  the  difference  between  the  precision  and 
volume  fire  support  capability  requirements  underlying  the  program  and  the  bal¬ 
listic  missile  defense  capability  requirement  that  resulted  in  the  truncation  of  the 
DDG-1000  program. 

•  Teaming  arrangements  between  shipyards  and  system  integrators,  and  the  rela¬ 
tionships  (contractual  and  technical)  between  teams. 

•  The  reduced  role  of  the  Navy  in  the  design  of  ship  and  mission  systems  and  the 
increased  responsibility  and  authority  of  industry. 

•  The  use  of  EDMs  as  subsystem  prototypes  and  test  beds  to  reduce  risk  and  refine 
ship  and  mission  system  design. 

•  The  consistent  underlying  policy  objective  of  maintaining  the  financial  and 
technical  viability  of  the  two  shipyards  capable  of  constructing  large  surface 
combatants. 


Conclusions 

The  original  program  baseline  (January  1998)  envisioned  a  32-ship  class  and  produc¬ 
tion  rates  of  up  to  three  per  year.  The  cost  estimate  at  the  time — $750  million  (in  FY 
1996  $;  approximately  $8 76  million  in  FY  2005  $)  for  the  fifth  ship — was  grossly 
understated  and  was  based  more  on  a  budget  constraint  and  the  need  to  maintain  two 
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viable  shipyards  than  on  any  cost  analysis.11  Over  time,  the  cost  analysis  was  performed 
and  a  more  realistic  estimate  was  eventually  adopted  at  Milestone  B.  It  is  interesting 
to  note  that  unit  cost  increased  between  the  January  1998  baseline  and  the  Novem¬ 
ber  2005  baseline  by  roughly  a  factor  of  three  ($876  million  to  $3.2  billion,  FY  2005 
$)  whereas  the  quantity  included  in  the  Milestone  B  baseline  decreased  by  a  factor  of 
three  (32  to  ten).  Unit  cost  is  more  sensitive  to  changes  in  quantity  when  procurement 
quantities  are  already  low:  Research  and  development,  overhead,  and  other  administra¬ 
tive  costs  are  spread  over  a  smaller  base;  and  economies  of  scale  (i.e.,  cost  improvement 
curve  effects)  shrink. 

In  this  case,  total  planned  quantity  was  reduced  by  70  percent  (ten  to  three  ships), 
which  can  be  expected  to  only  increase  unit  costs  dramatically,  all  other  things  being 
equal.  That  quantity  change  explains  upward  of  80  percent  of  the  PAUC  increase; 
much  of  the  rest  is  due  to  increases  in  scope  (work  content).  Other  factors  such  as 
technical  difficulties,  schedule  slips,  or  poor  performance  by  government  or  contractor 
personnel  do  not  play  a  role  here. 

The  quantity  change  resulted  from  the  2008  decision  of  senior  Navy  officials  to 
truncate  the  program  at  two  ships  (with  a  third  added  back  in  later).  That  decision, 
formally  incorporated  into  the  FY  2011  President’s  Budget,  was  based  on  the  percep¬ 
tion  of  an  emerging  ballistic  missile  threat  and  the  resulting  change  in  mission  priority 
from  precision  and  volume  fire  support  to  ballistic  missile  defense.  Remaining  cost  risk 
in  the  DDG-1000  program  and  affordability  concerns  contributed  to  the  decision  to 
base  that  ballistic  missile  defense  capability  on  the  DDG-51  platform  rather  than  on 
the  DDG-1000. 

DDG-1000  program  outcomes  were  driven  largely  by  factors  external  to  the  Pro¬ 
gram  Office.  At  Milestone  B,  there  was  a  mismatch  of  desired  performance  (required 
capabilities)  and  resources,  leading  to  chronic  underfunding  and  affordability  prob¬ 
lems.  The  Milestone  B  ADM  directed  that  the  Navy  fund  the  program  to  its  cost  esti¬ 
mate  rather  than  to  the  CAIG  estimate,  which  was  $4.1  billion  higher  over  the  period 
FY  2007-FY  2011.  In  the  FY  2008  and  subsequent  budgets,  the  Navy  did  not  fully 
fund  the  program  to  the  Program  Office’s  cost  estimate. 

It  is  important  to  note  that  the  DDG-1000  has  been  well  managed  by  a  small, 
dedicated  government  team  with  continuity.  EDMs  were  used  effectively  to  reduce 
risk.  Detail  design  was  largely  complete  (over  80  percent  drawing  complete)  before 
construction  began.  The  Program  Office  was  aggressive  in  contract  negotiations 
throughout,  protecting  Navy  interests.  Post-Milestone  B  metrics  generally  reflect  good 
program  execution,  with  relatively  small  variances  in  cost  and  schedule  metrics  to  date. 

Significant  cost  and  performance  uncertainty  remains  in  the  DDG-1000  pro¬ 
gram.  Ship  construction  has  been  under  way  for  only  a  little  over  a  year,  and  there  are 
limited  cost  data  on  which  to  base  new  estimates.  The  challenges  and  cost  of  system 


ii 


This  observation  was  made  by  several  officials  involved  in  the  early  (DD-21)  program. 
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integration,  including  both  ship  and  mission  systems,  have  yet  to  be  determined.  Other 
remaining  risks  include  software  development  and  integration,  hull  form  sea-keeping 
and  signature  requirements,  and  IPS  integration  and  test. 


Next  Steps 

Given  that  the  lead  ship  is  still  under  construction  with  many  significant  integration 
challenges  still  to  come,  program  execution  and  SCN  funding  should  be  carefully 
monitored.  It  would  also  be  useful  to  verify  the  claim  of  budget  instability  and  its  effect 
on  program  execution. 


CHAPTER  FOUR 


Joint  Strike  Fighter  (F-35) 


On  March  26,  2010,  the  Air  Force  notified  OSD  and  Congress  of  a  Nunn-McCurdy 
breach  in  the  JSF  program.  The  immediate  cause  of  the  breach  was  an  increase  in  the 
program’s  cost  estimates,  which  had  already  grown  substantially  from  the  2001  pro¬ 
gram  baseline.  The  resulting  57  percent  increase  in  PAUC  and  58  percent  increase  in 
APUC  triggered  the  Nunn-McCurdy  breach  and  associated  process. 


Program  Overview 

An  aging  fleet  of  fighter  aircraft  and  declining  budgets  prompted  DoD  to  establish  the 
Joint  Advanced  Strike  Technology  (JAST)  program  in  January  1994.  The  program  was 
created  to  mature  technologies  that  could  be  used  in  the  next  generation  of  affordable 
tactical  aircraft.  In  October  1994,  Congress  passed  legislation  that  merged  JAST  with 
the  Advanced  Short  Takeoff  and  Vertical  Landing  (ASTOVL)  program  managed  by 
the  Defense  Advanced  Research  Projects  Agency  (DARPA).  The  program  envisioned 
development  of  a  demonstrator  aircraft  with  the  potential  to  meet  the  requirements 
of  the  services  flying  fighters:  the  Air  Force,  Navy,  and  Marine  Corps.1  The  joint  pro¬ 
gram  would  be  managed  by  a  JPO,  with  leadership  rotating  between  the  Air  Force  and 
Navy,  and  would  seek  international  partners. 

A  concept  demonstration  phase  (CDP)  for  the  Joint  Strike  Fighter  program  began 
with  a  Joint  Initial  Requirements  Document  (JIRD)  promulgated  in  August  1995. 
This  JIRD  was  updated  in  1997  and  1998  and  a  draft  Joint  Operational  Requirements 
Document  (JORD)  was  signed  in  March  2000.  The  JROC  validated  the  JORD  in 
April  2000.  The  CDP  first  flight  took  place  in  September  2000,  and  the  flight  test  was 
completed  in  August  2001. 2 


1  See  “The  F-35  Lightning  II  History,”  web  page,  undated. 

2  Chronology  based  on  BG  John  L.  Hudson  (USAF ),  JSF  Program  Update,  November  2001,  and  SARs. 
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SDD  began  in  October  2001  with  the  approval  of  Milestone  B  by  the  Defense 
Acquisition  Board  and  subsequent  letting  of  an  SDD  contract  to  Lockheed  Martin. 
We  use  the  baseline  that  was  adopted  in  October  2001  in  our  analysis. 


JSF  Technical  and  Integration  Challenges 

The  JSF  is  inherently  difficult  to  develop.  It  is  the  first  aircraft  to  combine  stealth  signa¬ 
tures  with  supersonic  flight  and  the  ability  to  perform  short  takeoff  and  vertical  land¬ 
ings  (STOVL).  Mission  systems  integration  across  the  three  platforms  also  poses  new 
challenges,  and  the  JSF  contains  an  unprecedented  level  of  sensor  fusion,  requiring 
development  of  large  amounts  of  software  code.  Table  4.1  compares  F-35  variants  with 
other  advanced  aircraft  programs.  None  has  the  breadth  of  the  F-35. 

The  F-35  combines  four,  sometimes  competing,  characteristics  among  its  three 
variants.  Supersonic  flight  and  stealth  signature  are  required  on  all  variants,  with 
STOVL  required  for  the  F-35B  and  the  capability  to  operate  from  aircraft  carriers 
required  for  the  F-35C.  Only  recently,  with  the  development  of  the  F-22A  Raptor,  did 
we  see  a  supersonic  stealth  aircraft.  Before  the  F-22,  aircraft  were  designed  for  a  par¬ 
ticular  mission,  choosing  one  characteristic  for  primary  design  optimization.  The  F-117 
and  B-2  were  primarily  designed  for  stealth.  The  F-14,  F-15,  F-16,  and  F/A-18  were  pri¬ 
marily  designed  for  supersonic  flight.  The  AV-8B  was  primarily  designed  for  STOVL. 
Few  aircraft  are  carrier-capable.  Carrier  aircraft  operate  in  a  very  different  flight  enve- 


Table  4.1 

Historical  Aircraft  Characteristics 


Stealth 

STOVL 

Supersonic 

Carrier 

Capable 

F-14 

X 

X 

F-15 

X 

F-16 

X 

F/A-18 

X 

X 

AV-8B 

X 

F-117 

X 

B-2 

X 

F-22 

X 

X 

F-35A 

X 

X 

F-35B 

X 

X 

X 

F-35C 

X 

X 

X 
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lope  from  other  aircraft.  The  approach  to  a  carrier  landing  must  be  much  slower  than 
landing  on  a  hard-surface  runway.  This  requirement  entails  different  design  for  control 
surfaces,  which  must  be  larger  than  on  other  aircraft.  They  also  require  stronger  (and 
therefore  heavier)  airframes  and  landing  gear  than  aircraft  that  operate  only  from  land- 
based  runways.  The  upshot  of  these  significant  differences  means  that  the  carrier  take¬ 
off  and  landing  (CV)  version  of  the  aircraft  will  have  much  less  commonality  with  the 
other  variants  and  will  likely  have  important  cost  implications. 

The  CAIG  memo  summarizing  its  independent  cost  estimate  (ICE)  for  the  JSF 
Milestone  B  decision  in  October  2001  acknowledged  that  the  program  was  highly 
risky,  from  both  a  technological  and  a  schedule  perspective.3  The  SDD  Milestone  B 
program  schedule,  driven  by  the  need  to  develop  an  affordable  aircraft  to  replace  aging 
combat  aircraft,  was  aggressive  and  highly  concurrent,  with  the  first  flight  in  the  fourth 
quarter  of  2005,  just  48  months  after  the  SDD  contract  award,  and  with  over  600  air¬ 
craft  procured  by  the  end  of  initial  operational  test  and  evaluation  (IOT&E)  in  March 
2012. 4  This  would  amount  to  a  concurrency  of  about  25  percent  for  the  JSF.5  Con¬ 
currency  for  the  F-22,  an  equally  challenging  technology  program,  was  18  percent.6 
Figure  4.1  shows  certain  elements  of  this  concurrency.  Note  the  triangles  indicating 
critical  design  review  (CDR)  and  first  flight  and  the  steep  ramp-up  in  production 
before  the  completion  of  IOT&E  in  2012.  Other  high-technology  programs  may  have 
had  similar  issues  of  concurrency  (e.g.,  the  F/A-18,  F-22,  DDG-1000,  CVX  (undesig¬ 
nated  aircraft  carrier),  and  Joint  Tactical  Radio  System  radios  but,  in  the  time  allowed, 
we  were  unable  to  review  or  analyze  them. 

Affordability  is  also  a  critical  element  of  the  program,  and  the  baseline  cost  esti¬ 
mates  and  acquisition  strategy  relied  heavily  on  hypotheses  about  the  benefits  of  acqui¬ 
sition  reform  to  meet  cost,  schedule,  and  performance  objectives.  Affordability  was  to 
be  achieved  through  a  number  of  acquisition  process  improvements.  Before  Milestone 
B,  Cost  as  an  Independent  Variable  (CAIV)  processes  were  used  extensively  to  make 
cost  and  operational  performance  trades  to  maintain  affordability.  Commonality  ini¬ 
tiatives  for  the  airframe,  vehicle  systems,  propulsion,  and  mission  systems  were  also 
expected  to  contribute  to  affordability.  The  program  expected  technology  innovations, 
such  as  the  use  of  streamlined  testing  and  expanded  use  of  simulation,  to  contain 
program  costs.  Design-to-manufacturing  efficiencies  were  assumed  in  developing  the 
program’s  cost  and  schedule.  The  steep  ramp-up  to  high-quantity  production  also  con- 


3  “CAIG  Milestone  B  Independent  Cost  Estimate  (ICE)  for  Joint  Strike  Fighter  (JSF)  Program,”  October  24, 
2001.  Not  available  to  the  general  public. 

4  CAIG  Milestone  B  Independent  Cost  Estimate  (ICE)  for  Joint  Strike  Fighter  (JSF)  Program,  2001. 

5  Concurrency  is  generally  defined  as  the  amount  of  IOT&E  completed  before  entering  production  of  a  system. 

6  U.S.  General  Accounting  Office,  “Tactical  Aircraft:  Concurrency  in  the  Development  and  Production  of  F-22 
Aircraft  Should  Be  Reduced,”  GAO/NSAID  95-59,  Washington,  D.C.,  April  1995. 
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Figure  4.1 
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tributed  to  affordability  and  was  reflected  in  program  cost  estimates.  Finally,  stable 
program  funding  was  identified  as  a  key  element  of  program  affordability.7 


Approach 

RAND  took  a  top-down  approach  to  its  JSF  root  cause  analysis.  It  reviewed  a  number 
of  the  previous  JSF  program  assessments  to  look  for  common  themes  or  other  critical 
factors  driving  cost  and  schedule  performance.  The  team  also  reviewed  a  selection  of 
articles  discussing  the  JAST  and  JSF  program  histories.  Access  to  data  and  officials 
familiar  with  the  program  was  critical  to  identifying  relationships  and  thus  root  causes. 
RAND  analyzed  data  provided  by  many  sources,  though  this  report  relies  heavily  on 
SAR  data  because  these  reports  convey  DoD’s  official  public  cost  estimates  for  the 
program.  In  the  end,  as  the  team  diagrammed  the  story,  additional  levels  of  complex¬ 
ity  and  causality  were  revealed.  RAND’s  effort  benefited  from  access  to  a  wide  range 
of  JSF  program  materials  provided  by  the  JPO  and  Lockheed  Martin,  as  well  as  from 
extensive  conversations  with  Lockheed  Martin  and  JPO  staff.  CAPE  provided  addi¬ 
tional  materials. 


7  “CAIG  Milestone  B  Independent  Cost  Estimate  (ICE)  for  Joint  Strike  Fighter  (JSF)  Program,”  2001. 
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The  next  section  presents  a  program  overview  including  the  facts  on  JSF  program 
cost  and  schedule  from  the  baseline  to  the  breach.  That  section  is  followed  by  an  over¬ 
view  of  the  causes  of  this  cost  growth  and  schedule  slippage  and  a  more  detailed  dis¬ 
cussion  of  key  elements.  This  chapter  on  the  JSF  concludes  with  a  description  of  root 
causes  and  identification  of  potential  future  cost  and  schedule  concerns. 


Nunn-McCurdy  Breach 

Nunn  McCurdy  breaches  are  calculated  based  on  cost  growth  subsequent  to  the  Mile¬ 
stone  B  baseline  program.  The  JSF  breach  was  triggered  by  cost  growth  of  58  percent 
in  APUC  and  57  percent  in  PAUC.  The  program’s  schedule  delays  also  triggered  a 
Nunn-McCurdy  breach  that  resulted  from  delays  in  carrier  (CV)  first  flight,  delayed 
delivery  of  the  first  production  aircraft,  and  a  slip  in  the  date  for  completing  IOT&E.8 

Costs  Grew  in  Both  Development  and  Procurement 

The  program  is  still  in  development  and  the  early  phases  of  low  rate  initial  production 
(LRIP).  As  of  May  2010,  fewer  than  a  dozen  aircraft  had  been  produced,  and  thus  JSF 
unit  cost  is  a  mix  of  actual  costs  and  estimates.  Of  the  total  estimated  development  cost 
of  roughly  $45  billion,  77  percent,  or  almost  $35  billion,  had  been  funded  through 
FY  2009.  Of  the  total  estimated  procurement  cost  of  $193  billion,  only  3  percent, 
or  $6  billion,  had  been  funded  through  FY  2009,  with  the  remaining  program  costs 
based  on  estimates  of  future  procurement  costs.  Figure  4.2  shows  the  relative  amounts 
of  incurred  and  estimated  costs  in  constant  (BY  2002)  dollars. 

Although  just  3  percent  of  the  procurement  budget  has  been  spent,  procurement 
costs  represent  about  80  percent  of  the  total  cost  growth  from  the  program  baseline 
estimate,  with  20  percent  attributed  to  RDT&E.  That  is,  although  the  program  has 
spent  the  majority  of  its  budgeted  RDT&E  funds  in  its  almost  ten-year  RDT&E  pro¬ 
gram,  the  anticipated  costs  of  procuring  roughly  2,400  aircraft  over  more  than  20  years 
dwarf  RDT&E  costs. 

JSF  Costs  Rose  in  Two  Main  Spurts 

The  Nunn-McCurdy  breach  did  not  occur  in  one  jump.  As  can  be  seen  by  the  trend 
line  in  Figure  4.3,  the  program  had  two  big  jumps  that  bookend  a  period  of  more 
modest  growth.9 

The  first  growth  spurt  occurred  in  the  first  three  years  of  the  program  and  is 
shown  in  both  the  blue  and  pink  lines.  This  growth  reflects  a  reduction  of  400  in  the 


8  Michael  B.  Donley,  Secretary  of  the  Air  Force,  Memorandum  to  the  Honorable  Carl  Levin,  “APUC,  PAUC 
and  schedule  data  from  SAR  dated  December  31,  2009,”  March  25,  2010,  pp.  4-5  (unit  cost)  and  p.  8  (schedule). 

9  Data  for  this  section  come  from  the  F-35  SARs  unless  otherwise  stated. 
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Figure  4.2 

F-35  Acquisition  Funding 
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Figure  4.3 

Cost  Changes  from  Baseline 
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quantity  of  aircraft  being  purchased  by  the  Department  of  the  Navy  (red  line)  and  a 
serious  weight  growth  and  design  issue  that  was  discovered  as  the  first  prototype  was 
built.  After  this  initial  change,  U.S.  quantity  remained  stable  with  changes  in  ones  and 
twos  as  test  aircraft  were  canceled  to  pay  for  program  cost  growth. 

However,  weight  and  design  issues  continued  to  affect  the  program.  In  2002 
through  2003  and  2004,  Lockheed  Martin  (LM)  engaged  in  a  Blue  Ribbon  Action 
Team  (BRAT)  and  subsequent  18-month  STOVL  Weight  Attack  Team  (SWAT)  effort 
to  address  critical  weight  and  design  problems  that  cascaded  into  other  issues  includ¬ 
ing  parts  design  and  the  propulsion  system.  After  the  major  aspects  of  the  weight  and 
design  issue  were  addressed,  in  roughly  2004,  the  program’s  costs  grew  more  modestly 
until  a  second  big  jump  in  2008-2009.  This  second  jump  reflects  changes  in  the  offi¬ 
cial  DoD  estimate  of  expected  program  costs  and,  more  specifically,  anticipated  future 
cost  growth. 

Estimated  Cost  Changes10 

RAND  obtained  four  sets  of  data  showing  different  categories  of  cost  growth.  The  data 
are  not  comparable;  each  organization  binned  costs  differently  and  some  presented  the 
data  as  program  costs  whereas  others  showed  only  unit  costs.  However  the  data  do  pro¬ 
vide  some  explanatory  value.  Here  we  discuss  the  SAR  data;  the  data  from  the  other 
organizations  appear  in  Appendix  D. 

Figure  4.4  provides  SAR  cost  variance  data  in  six  categories:  estimating,  eco¬ 
nomic,  schedule,  engineering,  quantity,  and  support.  These  data,  which  are  not  pre¬ 
sented  in  terms  of  unit  costs  but  rather  in  terms  of  total  program  cost,  identify  sched¬ 
ule  delays  as  the  primary  driver  of  cost  growth  in  RDT&E  (77  percent  for  schedule 
individually)  followed  by  estimating  and  engineering  (6  and  30  percent,  respectively). 
For  procurement,  which  is  a  far  larger  figure  than  RDT&E,  estimating  accounts  for 
94  percent  of  the  cost  growth.  The  next  largest  factor  driving  cost  is  quantity;  this  refers 
to  the  400  aircraft  reduction  in  the  Navy’s  program.  Quantity  is  actually  assigned  a 
negative  effect  on  cost  growth  equal  to  about  one-third  the  effect  of  estimating  growth 
because  it  reduced  the  overall  cost  of  procurement.  Engineering,  support,  and  schedule 
are  the  remaining  drivers  of  procurement  cost  growth,  though  much  smaller  in  scale 
than  estimating  or  quantity. 

Data  from  the  other  cost  estimates  appear  in  Appendix  D.  Although  the  data 
do  not  lend  themselves  to  comparison,  they  do  convey  a  common  message,  summa¬ 
rized  in  Table  4.2.  They  generally  agree  that  program  cost  growth  was  driven  in  large 
measure  by  optimistic  cost  and  schedule  estimates,  program  weight  and  design  issues, 
problems  with  design  control  and  parts  production,  and  growth  in  labor  rates  and  raw 
material  costs.  It  is  also  true  that  the  reduction  of 400  aircraft  in  the  Navy  Department 
buy  affected  unit  costs  and  thus  the  APUC  and  PAUC  calculations. 


10 


The  organizations  are  CAIG/CAPE,  Lockheed  Martin,  JPO,  and  AFCAA. 
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Figure  4.4 

Explanations  of  Cost  Growth  from  SARs,  2001-2009  ($  millions) 
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Table  4.2 

Identified  Cost  Growth  Drivers 

Estimating 

Navy  Quantity 
Reduction 

Weight  and 
Design 

Cost  of 
Materials 

Rates 

Propulsion 

System 

Schedule 

Delays 

SAR  X 

LM 

X 

X 

X 

JPO 

X 

X 

X 

X 

CAIG  X 

X 

SOURCES:  SARs,  December  31,  2002  through  December  31,  2009;  LM,  JPO,  and  CAIG  briefings  to  the 
PARCA  team  in  April  and  May  2010. 


JSF  Program  Schedule  Slipped  Repeatedly 

The  program  schedule  slipped  repeatedly  after  2001,  with  average  milestone  delays  of 
two  years  through  first  flight.  A  key  early  delay  resulted  from  aircraft  weight  growth 
and  design  problems  that  delayed  CDR  by  about  two  years.  During  the  ensuing  design 
review,  the  STOVL  variant  was  identified  as  the  critical  design  challenge,  and  both 
conventional  takeoff  and  landing  (CTOL)  and  STOVL  variants  went  through  CDR 
together  in  February  2006.  Table  4.3  shows  the  schedule  slippage  calculated  from  data 
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Table  4.3 

Current  Estimate  Dates  from  2001  SAR  Compared  with  2009  SAR 


Critical  Design  Review 

December  2001 

December  2009 

Slip  (Years) 

CDR CTOL 

April  2004 

February  2006 

1.8 

CDR STOVL 

October  2004 

February  2006 

1.3 

CDR  CV 

July  2005 

June  2007 

1.9 

First  flight  CTOL 

November  2005 

December  2006 

1.1 

First  flight  STOVL 

April  2006 

June  2008 

2.2 

First  flight  CV 

January  2007 

May  2010 

3.3 

U.S.  Marine  Corps  IOC 

April  2010 

March  2012 

1.9 

U.S.  Air  Force  IOC 

June  2011 

March  2013 

1.8 

Completed  IOT&E 

March  2012 

April  2016 

4.1 

U.S.  Navy  IOC 

April  2012 

March  2015 

2.9 

in  the  December  2001  and  December  2009  SARs.  The  scheduled  first  flight  of  the  CV 
variant  occurred  in  June  2010, 11  illustrating  the  point  that  much  of  the  schedule  has  yet 
to  be  completed  and  thus  has  the  potential  to  slip  further.  Although  the  average  major 
milestone  schedule  slip  to  date  is  slightly  under  two  years,  the  slippage  of  IOT&E — 
the  date  that  SDD  will  be  considered  complete — has  already  reached  four  years  and 
with  additional  delays  in  first  flight  seems  likely  to  grow. 

These  schedule  delays  affected  the  production  schedule.  Program  affordability 
depended  on  a  quick  ramp-up  to  a  high  rate  of  U.S.  production  (Figure  4.1)  (almost 
200  aircraft  per  year)  within  six  years  of  first  flight.  As  of  the  December  2009  SAR,  the 
ramp-up  in  U.S.  production  will  take  ten  years,  until  2016,  and  reach  only  130  U.S. 
aircraft  per  year.12  Figure  4.5  shows  the  shift  in  the  U.S.  procurement  plan  from  2001 
to  2009 — slower  ramp-up  and  lower  production  levels.  The  dark  blue  line  shows  the 
2001  baseline  program  and  the  dark  red  line  shows  the  2009  SAR-reported  program. 
The  other  lines  trace  the  incremental  annual  shifts  in  schedule  and  quantity.13 

International  procurement  will  improve  the  speed  of  the  ramp-up  in  production 
but  will  not  bring  it  back  to  the  original  plans.  As  of  2009,  peak  production  including 
international  production  will  occur  in  2019,  at  over  200  aircraft  per  year.14  Figure  4.6 


11  First  flight  was  previously  scheduled  for  May  2010  and  occurred  on  June  7,  2010,  instead.  See  Lockheed 
Martin,  “Press  Releases  for  F-35  Lightning  II,”  various  dates. 

12  SARs  of  2001-2009. 

13  SAR  schedules. 

14  Annual  cost  estimating  files  are  from  the  JSF  collaboration  website.  U.S.  data  are  from  2001  and  2009  SARs. 
International  data  are  from  Lockheed  Martin  presentations  to  Gary  Bliss,  April  23,  2010. 
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Figure  4.5 

Procurement  Plan  Changes  Through  Each  SAR 


Fiscal  year 
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Figure  4.6 

Procurement  Plan  with  International  Procurement  Added 


Fiscal  year 

SOURCES:  Annual  cost  estimating  files  are  from  the  JSF  collaboration  website.  U.S.  data  are  from  2001 
and  2009  SARs.  International  data  are  from  Lockheed  Martin  presentations  to  Gary  Bliss,  April  23,  2010. 
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shows  the  production  schedule.  The  blue  line  shows  the  original  U.S.  buy.  The  red  line 
shows  the  current,  stretched  out,  U.S.  buy,  and  the  green  line  shows  additional  foreign 
aircraft  as  of  May  2010. 

These  international  sales  are  not  included  in  the  Nunn-McCurdy  calculations  of 
PAUC  and  APUC,  which  focus  only  on  the  U.S.  element  of  the  program.  It  is  likely 
that  inclusion  of  planned  international  procurement  would  change  the  PAUC  calcula¬ 
tion.  Although  the  increase  in  production  quantities  should  be  good  for  efficiency  and 
for  reducing  unit  costs,  such  an  increase  assumes  that  Lockheed  Martin  can  success¬ 
fully  manage  the  complexity  of  the  program  including  the  steep  ramp-up  for  inter¬ 
national  production,  the  integration  of  many  foreign  manufacturing  entities,  and  the 
production  of  an  additional  6-12  variants  as  each  country’s  model  is  slightly  different. 
The  effect  of  changes  in  U.S.  airframe  commonality  standards  and  the  further  influ¬ 
ence  on  commonality  cost  savings  from  the  multiple  variants  remains  to  be  seen. 

In  response  to  cost  growth  and  schedule  slips,  the  program  was  re-baselined 
twice,  once  in  March  2004  and  a  second  time  in  March  2007. 15  At  both  these  points 
the  schedule  was  also  re-baselined.  It  is  anticipated  that  another  re-baseline  will  occur 
in  the  wake  of  the  Nunn-McCurdy  breach. 


Findings  Narrative 

The  PARCA  law  specifies  a  set  of  criteria  to  be  assessed  in  DoD’s  program  review 
following  a  Nunn-McCurdy  breach.  These  divide  into  baseline  issues  and  execution 
issues.  For  the  JSF,  the  RAND  team  further  divided  these  elements  into  issues  related 
to  development  and  issues  related  to  procurement.  In  Table  4.4,  the  reader  will  find 
that  blue  shaded  cells  speak  to  problems  in  RDT&E  and  green  shaded  cells  relate  to 
procurement. 

The  reader  should  remember  that  this  analysis  is  focused  on  root  causes,  so  this 
review  does  not  capture  all  the  issues  that  may  have  come  up  in  the  program.  We 
used  the  SARs  to  determine  dates,  though  it  is  hard  to  be  precise  about  them,  in  part 
because  root  causes  can  lag  manifestation  of  a  problem. 

Baseline  Issues 

As  noted  in  the  research  approach,  RAND’s  analysis  built  on  a  number  of  prior  and 
ongoing  studies  on  JSF  cost  and  schedule  issues.  In  response  to  Lockheed  Martin’s 
early  problems  with  JSF  weight  growth  and  design  configuration  and  in  response  to 


SARs  of  December  2004  and  December  2007  report  APBs  as  follows:  APB  March  17,  2004,  and  APB 
March  30,  2007. 


Table  4.4 

PARCA  Root  Cause  Narrative  Matrix  for  the  JSF 


Year  from  MS  B  and  Fiscal  Year 

B 

2001 

1 

2002 

2 

2003 

3 

2004 

4 

2005 

5 

2006 

6 

2007 

7 

2008 

8 

2009 

Baseline  issues 

Unrealistic  estimates  for 
cost  or  schedule 

Acquisition 

strategy; 

concept 

demonstrator 

X 

X 

APB  breach 

Immature  technology, 
excessive  manufacturing, 
integration  risk 

Integration 

challenges 

X 

X 

Unrealistic  performance 
expectations 

Systems 
engineering 
for  weight; 
commonality 

X 

X 

Execution  issues 

Changes  in  procurement 
quantity 

X 

U.S.  Navy 
reduced  ~  400 

X 

X 

X 

X 

X 

X 

X 

Inadequate  funding/ 
funding  instability 

Unanticipated 
design,  engineering, 
manufacturing,  or 
technical  issues 

Engine 

Aircraft  weight 
growth; 
common  parts; 

Changed 
support 
estimate; 
test  program 
delays 

Materials;  wing; 
support 

Production 

Tooling;  test 
manufacturing; 
lift  fan 

X 

Rates; 

manufacturing 
plan;  materials; 
spares 

schedule;  rates 

Poor  performance  of 
government  or  contract 
personnel 

CAIV  process 

X 

X 

X 

EVMS  issue 

X 

X 

X 
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subsequent  concerns,  DoD  conducted  a  number  of  its  own  investigations  into  JSF  cost 
growth  and  schedule  slippage.  These  included  the  following:16 

•  Joint  Estimating  Team  (JET  1)  in  2008 

•  Joint  Estimating  Team  (JET  2)  in  2009 

•  Independent  Manufacturing  Review  Team  (IMRT)  in  2009. 

These  studies  variously  identified  problems  with  weight  and  design,  quantity 
changes,  materials  costs,  labor  rates,  and  engineering  as  the  drivers  of  cost  and  sched¬ 
ule  performance  issues.  RAND  explored  a  number  of  these  findings  to  get  a  better 
understanding  of  the  root  causes  of  these  problems. 

Unrealistic  Baseline  Estimates  for  Schedule  and  Cost.  The  requirement  to  replace 
aging  fighters,  the  need  for  affordability,  and  the  belief  that  significant  cost  savings 
would  be  realized  from  implementation  of  acquisition  reform  practices  all  contributed 
to  an  overly  optimistic  schedule  and  cost  estimate. 

The  JSF  Schedule  Was  More  Aggressive  Than  the  F-22  Schedule.  Table  4.5  com¬ 
pares  the  JSF  baseline  schedule  to  those  of  other  fighters.  Although  the  JSF  baseline 
program  schedule  allowed  more  time  than  most  aircraft  between  contract  award  and 
first  flight,  and  between  contract  award  and  IOC  (March  2012),  these  time  periods 
were  less  than  those  for  the  F-22.  The  original  schedule  for  STOVL  from  contract 
award  to  IOC  was  tighter  than  for  both  F/A-18E/F  and  F-22. 

Milestone  B  cost  estimates  of  both  the  JPO  and  the  CAIG  understated  program 
costs.17  These  estimates  were  based  on  legacy  aircraft  data  including  both  the  F/A- 
18E/F  and  the  F-22.  However,  only  a  few  F-22  development  units  had  been  built  at  the 
time  of  the  cost  estimate  in  2001,  and  thus  these  estimates  did  not  have  the  benefit  of 
meaningful  knowledge  about  F-22  procurement  costs.  More  recent  CAPE  estimates 
incorporated  actual  cost  data  from  the  F-22  program,  contributing  to  an  increase  in 
the  JSF  procurement  cost  estimate.18 

A  key  contributor  to  the  understated  estimates  is  insufficient  margin  for  weight 
growth.  The  Milestone  B  CAIG  estimate  included  a  6  percent  margin  for  aircraft 
weight  growth  based  on  legacy  aircraft  experience.19  JSF  actual  weight  growth  differed 
depending  on  the  variant.20  This  is  important  because  parametric  cost  estimating  is 
based  on  aircraft  weight,  and  thus  an  underestimate  of  the  aircraft  weight  carries  over 
into  all  aspects  of  the  cost  estimates. 


16  Joint  Estimating  Teams  were  composed  of  cost  estimating  experts  from  CAIG,  OSD,  and  the  other  services. 

17  These  estimates  were  within  3  percent  of  each  other  per  Milestone  B  estimates. 

18  Christine  H.  Fox,  Director  Cost  Assessment  and  Program  Evaluation,  OSD,  testimony  before  the  U.S.  Senate 
Committee  on  Armed  Services,  March  11,  2010. 

19  “CAIG  Milestone  B  Independent  Cost  Estimate  (ICE)  for  Joint  Strike  Fighter  (JSF)  Program,”  2001. 
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SARs,  2001-2009. 
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Table  4.5 

Comparison  of  Legacy  Aircraft  Program  Months  to  First  Flight  and  IOC 


Aircraft 

Contract 

Award 

First  Flight 

DT&E End 

IOC 

Months  Months 

from  Contract  from  Contract 
Award  to  First  Award  to 
Flight  IOC 

A-10 

March  1973 

February  1975  June  1977 

October  1977 

23 

55 

F-14 

January  1969 

December 

1970 

September 

1974 

December 

1973 

23 

59 

A- 6 

December 

1957 

April  1960 

October  1963 

February  1963 

28 

62 

F-111 

November 

1962 

December 

1964 

July  1972 

April  1968 

25 

65 

F-16 

January  1975 

December 

1976 

January  1979 

June  1980 

23 

65 

F-15 

January  1970 

July  1972 

March  1976 

September 

1975 

30 

68 

AV-8B 

August  1979 

November 

1981 

December 

1984 

August  1985 

27 

72 

F/A-18A/B 

January  1976 

November 

1978 

March  1982 

March  1983 

34 

86 

F/A-18E/F 

June  1992 

November 

1995 

April  1999 

September 

2001 

41 

111 

JSF/AF 

baseline 

October  2001 

November 

2005 

March  1912 

June  2011 

49 

117 

JSF/AF 

current 

October  2001 

December 

2006 

April  2016 

March  2013 

62 

137 

F-22 

August  1991 

September 

1997 

December 

2005 

December 

2005 

73 

172 

SOURCES:  Program  SARs  and  Institute  for  Defense  Analyses  (IDA),  "Assessing  Acq.  Schedules,"  P-2105, 
February  1989. 


Immature  Technology,  Excessive  Manufacturing  and  Integration  Risk. 

Technology  integration  experience  was  immature.  Although  the  key  structural 
technologies — stealth,  supersonic,  STOYL,  and  carrier-capable — used  on  the  JSF  had 
all  been  demonstrated  on  other  aircraft,  integration  of  the  STOVL  and  carrier-based 
elements  with  stealth  and  supersonic  elements  was  new  and  complicated.  Trade-offs 
are  always  necessary  when  designing  aircraft,  but  combining  different  characteristics, 
as  was  the  case  with  the  F-35,  creates  countervailing  trades.  In  a  traditional  aircraft, 
for  example,  if  weight  increases,  designers  can  take  several  measures  to  mitigate  the 
increase,  including  increasing  engine  thrust,  speed  required  for  takeoff,  the  rotation 
angle,  and  the  wing  area  (which  also  increases  weight). 
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All  of  these  methods  can  be  traded  off  against  each  other  to  achieve  an  optimal 
design.  Unlike  a  conventional  aircraft  that  has  more  options  to  overcome  weight,  the 
only  characteristics  a  STOVL  aircraft  can  balance  is  weight  and  thrust.  So  the  only 
option  that  designers  have  to  overcome  an  increase  in  weight  is  to  increase  engine 
thrust.  However,  generating  more  thrust  with  a  given  engine  size  requires  more  airflow, 
which  requires  larger  engine  inlets.  This  action  conflicts  with  stealth  measures,  which 
limit  the  size  of  engine  inlets  to  reduce  aircraft  signature.  Table  4.6  shows  some  of  the 
challenging  design  characteristics  that  an  aircraft  like  the  F-35  must  overcome. 

Manufacturing  and  integration  risks  were  acknowledged  but  not  accounted 
for.  The  government  acknowledged  at  Milestone  B  that  the  JSF  was  a  risky  program 
with  an  aggressive  schedule.  The  CARD  at  Milestone  B  highlighted  the  complex  inte¬ 
gration  challenge  facing  Lockheed  Martin  and  its  concern  about  the  company’s  abil¬ 
ity  to  address  these  challenges.  Indeed,  the  draft  CARD  supporting  the  Milestone  B 
decision  laid  out  185  risk  issues  with  50  rated  low,  89  medium,  and  46  high.  The  list 
included  four  risk  issues  that  are  now  quite  familiar: 

•  engine  operability — high  risk 

•  EDM  schedule  milestones — medium  risk 

•  weight  assessment  risk — medium  risk 

•  weight  definitions  and  loss  of  growth — medium  risk 

It  is  clear  in  hindsight  that  the  contractor  was  not  ready  to  address  the  program’s 
manufacturing  and  integration  challenges.  Two  factors  identified  as  contributors  to 
Lockheed’s  inability  to  address  the  program’s  challenges  stand  out: 


Table  4.6 

Design  Trades  and  JSF  Characteristics 


Stealth 

STOVL 

Supersonic 

Engine  inlets 

As  small  as  possible  to 
reduce  signature 

Large  to  allow  as  much 
airflow  as  possible 

Specific  shape  for  supersonic 
shockwaves 

Fuel  storage 

All  internal  to  keep 
signature  small 

As  little  weight  as 
possible  to  allow  STOVL 
operations 

As  much  fuel  as  possible  for 
afterburning  engines 

Aircraft  shape 

Specific  shape  for 
reduced  signature 

Shape  dictated  by 
weight  distribution 

Shape  dictated  by 
supersonic  flight  and  speed 
regime  transitions 

Materials 

Allow  for  aircraft  skin 
to  assist  in  stealth 
operations 

Light  skin  to  allow  for 
vertical  operations 

Strong  skin  to  allow  for  sub-, 
trans-,  and  supersonic  flight 
transitions 
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•  the  fact  that  a  concept  demonstrator  was  not  an  adequate  basis  for  SDD 

•  a  sequential  approach  to  aircraft  design,  focused  on  the  CTOL  version,  that 
reduced  emphasis  on  other  variants  notwithstanding  the  fact  that  parts  common¬ 
ality  was  a  critical  element  of  affordability. 

We  discuss  these  two  points  below. 

A  concept  demonstrator  may  not  have  been  sufficient  preparation  for  exe¬ 
cuting  SDD.  The  theoretical  debate  around  demonstration  versus  prototype  can  be 
thought  of  as  a  scale  with  full  production  prototype  on  the  far  left  and  proof  of  concept 
demonstrator  on  the  far  right.  Figure  4.7  shows  where  different  types  of  preproduction 
aircraft  can  fall. 

•  full  production  prototype:  aircraft  built  using  procedures,  equipment,  and 
technology  as  would  appear  on  a  full  production  model 

•  technology  prototype:  aircraft  built  to  use  the  production  technology  but  not 
produced  in  the  same  way  as  the  full  production  models  would  be  produced 

•  key  technology  demonstrator:  aircraft  built  to  show  key  pieces  of  technology  in 
operation,  though  the  aircraft  itself  would  not  represent  a  production  model;  only 
key  technology  would  be  tested 

•  proof-of-concept  demonstrator:  aircraft  that  does  not  represent  the  technology 
intended  for  use  in  a  production  model  but  that  demonstrates  that  operational 
concepts  are  possible. 

The  YF-16,  which  was  eventually  developed  into  the  F-16  Fighting  Falcon,  falls 
on  the  prototype  side  of  the  scale.  The  X-35,  which  was  developed  into  the  F-35  Light¬ 
ning  II,  falls  on  the  demonstrator  side  of  the  scale,  close  to  the  proof-of-concept  demon¬ 
strator.  By  building  an  YF-16  prototype,  General  Dynamics/Lockheed  Martin  learned 
about  what  could  and  could  not  be  done  in  the  manufacturing  phase. 

Lockheed  Martin  demonstrated  its  ability  to  perform  supersonic  flight  and  ver¬ 
tical  landing  within  a  stealth  aircraft  and  demonstrated  plans  related  to  commonal¬ 
ity,  modularity,  and  affordability.  It  did  not  demonstrate  its  ability  to  translate  these 

Figure  4.7 

Scale  of  Prototype  to  Demonstrator 

Full  Key 

production  technology 

prototype  demonstrator 
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Technology 

prototype 


Proof  of 
concept 
demonstrator 
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capabilities  into  a  production  aircraft.21  In  fact,  although  the  aircraft  did  demonstrate 
capabilities,  it  did  not  do  so  using  key  technologies.  Additionally,  the  simplified  wing 
box  that  was  flown  on  the  X-35  had  to  be  redesigned. 

A  prototype,  on  the  other  hand,  would  have  required  that  Lockheed  Martin  build 
an  aircraft  that  was  as  much  like  a  production  model  as  possible,  with  more  elements 
integrated  so  that  more  cost  and  performance  trade-offs  would  have  been  done  in 
advance  of  SDD.  In  fact,  it  appears  that  two-thirds  of  the  design  was  finalized  after 
SDD  contract  award.22  It  is  not  clear  that  building  a  prototype  would  have  avoided 
the  weight  and  design  issues,  but  it  is  clear  that  a  demonstrator  does  not  provide  much 
guidance  on  produceability. 

In  the  case  of  the  JSF,  the  tenets  of  acquisition  reform,  the  press  for  affordability, 
and  the  drive  to  replace  aging  aircraft  led  the  government  to  use  a  technology  demon¬ 
stration  approach  to  identifying  its  JSF  prime  contractor.  Fiad  a  good  prototype  effort 
been  undertaken  before  Milestone  B,  perhaps  two  years  of  post-Milestone  B  delay 
would  have  been  avoided.  At  the  time,  however,  the  need  for  a  new  replacement  fighter 
led  DoD  to  move  quickly  and  use  a  concept  demonstration  approach.  Cost  avoidance 
is  more  difficult  to  ascertain  as  there  would  have  been  additional  costs  associated  with 
a  prototype  and  a  subsequent  redesign,  along  the  lines  of  those  associated  with  the  cur¬ 
rent  program’s  SWAT  program. 

Engineering  Strategy  Focused  on  CTOL.  The  initial  concept  of  developing  a 
STOVL  aircraft  for  the  U.S.  Marine  Corps,  then  simply  taking  out  the  lift  fan  and 
adding  an  additional  fuel  tank  for  the  USAF  CTOL  variant,  was  abandoned  when 
the  CTOL  variant  was  chosen  for  development  first.  The  strategy  of  starting  with  the 
CTOL  variant — a  result  of  a  focus  on  the  need  for  an  F-16  replacement  rather  than 
on  engineering  feasibility — may  also  have  contributed  to  a  lack  of  focus  on  aircraft 
weight.  Standard  engineering  practices  usually  lead  to  the  development  of  the  most 
difficult  system  first.  Engineering  principles  dictate  that  the  more  challenging  STOVL 
aircraft  should  have  been  designed  first,  with  limits  relaxed  if  possible  for  the  CTOL 
and  CV  variants.23 

By  focusing  on  the  less  complex  CTOL  system  before  the  more  complicated  dif¬ 
ficult  STOVL  one,  Lockheed  Martin  did  not  fully  tackle  the  most  difficult  systems 
engineering  challenges  first.  This  may  have  contributed  to  delays  in  addressing  weight, 
which  grew  so  much  that  a  complete  redesign  effort  was  required.  In  this  redesign 
process,  the  more  restrictive  requirements  for  STOVL  weight  were  addressed,  but  had 
the  STOVL  been  the  initial  focus  of  attention,  the  weight  issues  might  have  been  dealt 
with  sooner. 


21  F-35  Joint  Program  Office,  “JPO  Cost  Growth  Summary,”  JSFSDD-#152864,  January  8,  2009. 

22  Telephone  conversation  with  JPO  representative,  May  11,  2010. 

23  The  United  States  also  had  the  least  experience  with  the  STOVL  design,  as  STOVL  Harriers  previously  manu¬ 
factured  in  the  United  States  were  originally  designed  in  the  United  Kingdom  and  adapted  for  the  U.S.  market. 
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Unrealistic  Performance  Expectations.  In  a  corollary  to  the  point  made  above 
regarding  the  lack  of  appreciation  of  the  difficulties  of  technology  integration,  there 
appears  to  have  been  excessive  optimism  as  to  the  feasibility  of,  and  savings  to  be 
gained  by,  commonality  of  design  and  parts  and  utilization  of  new  methods  for  manu¬ 
facturing  and  testing.  These  production-focused  issues  had  implications  for  aircraft 
performance  as  well  as  for  program  execution.  Implications  for  performance  appeared 
primarily  in  the  weight  growth  and  subsequent  efforts  to  cut  weight  to  maintain  a 
viable  STOVL  variant.  Implications  for  program  execution  are  discussed  in  the  fol¬ 
lowing  sections. 

Execution  Issues 

Quantity  Reduction.  Addressing  the  first  of  the  execution  issues,  quantity,  is 
straightforward.  The  U.S.  Navy  cut  its  procurement  quantity  by  400  in  the  first  year 
of  the  program.  Although  the  JPO  assigns  this  action  as  contributing  4  percent  to 
the  38  percent  cost  growth,  the  crisis  over  weight  growth  and  the  size  of  the  overall 
anticipated  fleet  make  this  a  point  to  note  but  not  a  driver  of  cost  growth.24  As  noted 
above,  the  Nunn-McCurdy  breach  is  calculated  just  on  the  U.S.  buy  and  thus  misses 
any  implications  for  Lockheed’s  unit  cost  resulting  from  foreign  sales.  This  is  likely  to 
be  a  significant  omission  for  the  JSF,  which  is  expected  to  have  a  large  foreign  sales 
component. 

Program  Funding  Stability.  One  affordability  factor  that  the  CAIG  MS  B  esti¬ 
mate  identified  as  critical25  and  that  DoD  was  able  to  control  was  program  funding 
stability.  Despite  turbulence  in  other  dimensions,  funding  as  measured  by  the  differ¬ 
ence  between  the  President’s  Budget  request  and  the  appropriation  was  remarkably 
stable,  with  the  biggest  differences  related  to  funding  for  a  second  engine  program 
that  DoD  frequently  deleted  and  Congress  restored.  Figure  4.8  shows  the  program 
funding  from  FY  2002  through  FY  2010.  The  FY  2005  reduction  is  due  to  re-planned 
schedule  changes,  the  FY  2006  reduction  is  due  to  excessive  risk  before  CDR,  and 
the  FY  2007-2010  net  increase  is  due  to  congressional  funding  for  the  F-136  alterna¬ 
tive  engine  development.  Unfortunately,  funding  stability  was  not  enough  to  prevent 
schedule  delays  and  cost  growth. 

Unanticipated  Design,  Engineering,  Manufacturing,  or  Technical  Issues.  Many 
technical  and  integration  problems  were  encountered  in  executing  the  SDD  program. 
As  described  above,  the  program  faced  an  aggressive  schedule  and  keen  focus  on 
affordability,  and  successful  SDD  required  a  clean  and  efficient  transition  from  con¬ 
cept  demonstration  to  test  and  production.  Several  factors  impeded  this  clean  and  effi¬ 
cient  transition:  A  strategy  focused  on  the  CTOL  variant  has  already  been  discussed; 
other  factors  included  insufficient  engine  capability,  insufficient  attention  to  weight 


24  “jpQ  Cost  Growth  Summary,”  2009. 

25  “CAIG  Milestone  B  Independent  Cost  Estimate  (ICE)  for  Joint  Strike  Fighter  (JSF)  Program,”  2001. 
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Figure  4.8 
Program  Funding 


SOURCES:  Data  are  for  SARs,  2001-2009,  and  JPO  annual  Congressional  Summaries,  provided  by  JPO, 

May  2010. 
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growth  and  design  control,  reductions  in  design  commonality,  incomplete  capture  of 
efficiencies  from  design-to-manufacture  initiatives  and  the  planned  test  program,  and 
cascading  manufacturing  and  production  issues  resulting  from  the  weight  and  design 
problems  and  consequent  schedule  delays. 

Engine  Issues.  Changes  in  the  engine  contributed  to  the  weight  growth  of  the  JSF. 
Original  plans  called  for  JSF  to  use  the  same  engine  as  the  F-22 — the  F-119  engine. 
However,  the  F-119  proved  to  be  underpowered  for  the  performance  desired  of  the 
F-35,  so  the  F-119  engine  was  altered  to  generate  more  thrust  and  became  the  F-135 
engine.  By  enlarging  the  F-119  engine  into  the  F-135  engine,  engineering  issues  such 
as  shaft  length  and  efficiency  had  to  be  dealt  with.  However,  the  increase  in  thrust  also 
lead  to  an  increase  in  the  engine  size  by  a  reported  1.5  inches  in  diameter.  This  small 
change  in  the  engine  generated  a  need  to  redesign  the  airframe,  which  in  turn  changed 
everything  from  aerodynamics  to  stealth  signature,  all  of  which  needed  to  be  re-base- 
lined.  This  engine  issue  also  indicates  lack  of  integration  across  the  major  contractors, 
which  was  Lockheed’s  responsibility  as  the  prime  contractor.26 


26  The  project  team  did  not  interview  Pratt  &  Whitney  and  therefore  all  information  was  received  by  the  project 
team  secondhand.  Engine  information  was  obtained  through  meetings  and  discussions  with  Lockheed  Martin 
on  April  23,  2010,  with  JPO  on  April  20,  2010,  and  with  AFCAA  on  April  16,  2010.  Those  discussions  did  not 
yield  much  insight  into  the  engine  issues  that  were  experienced  by  Pratt  &  Whitney. 
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Insufficient  Attention  to  Weight  Growth  and  Design  Control.  Aircraft  design 
execution  problems  and  weight  growth  caused  a  substantial  direct  growth  in  RDT&E 
costs  with  residual  impacts  on  procurement  costs.  Lack  of  attention  to  design  and 
manufacturing  integration  led  to  weight  growth  in  the  CTOL  variant.  This  became 
apparent  in  2002-2003  and  was  the  focus  of  a  BRAT  and  subsequently  an  18-month 
redesign  effort  SWAT. 

Commonality  Reduced  in  Airframe.  Commonality  is  another  area  in  which  cost 
savings  were  not  as  robust  as  initially  anticipated  and  may  have  generated  consider¬ 
ably  more  friction  in  the  process  than  expected.  As  stated  in  the  CAIG  ICE  memo  at 
Milestone  B,  maintaining  commonality  was  considered  critical  to  affordability.27  The 
expectation  behind  use  of  common  parts  is  that  they  can  reduce  the  costs  of  design  and 
production  as  well  as  long-run  support  costs.  Common  parts  need  only  be  designed 
once,  need  only  one  set  of  tooling,  and,  through  application  of  quantities  of  scale,  can 
reduce  O&S  costs,  ffowever,  the  three  variants  proved  different  enough  that,  particu¬ 
larly  with  the  airframe,  design  of  common  parts  proved  less  practicable  than  originally 
planned.  The  uniqueness  or  commonality  of  parts  is  measured  by  the  weight  of  the 
aircraft  in  the  tradition  of  parametric  cost  estimating.  The  greatest  growth  occurred 
during  the  period  Lockheed  Martin  worked  to  resolve  the  weight  growth  and  design 
issues,  during  which  time  commonality  was  given  less  priority  than  simply  designing 
an  aircraft  that  could  meet  performance  specifications.28  Once  the  weight  issue  was 
under  control  (roughly  2004),  the  share  of  unique  parts  as  a  percentage  of  airframe 
unit  weight  stayed  high  through  the  fifth  year  of  the  program  (2006),  presumably  as 
the  program  was  resolving  design  issues  for  component  parts,  ffowever,  since  about 
2006,  Lockheed  Martin  has  identified  opportunities  to  increase  commonality  slightly. 

Commonality  in  other  areas  of  the  aircraft,  such  as  in  propulsion  and  mission 
systems,  has  largely  remained  as  expected,  at  least  to  date.  The  impact  of  changes  in 
airframe  commonality  in  procurement  and  O&S  accounts  is  largely  still  to  be  deter¬ 
mined  because  only  a  few  aircraft  have  been  produced  so  no  actual  costs  are  available 
on  production  manufacturing  or  on  O&S  where  significant  savings  are  expected. 

Incomplete  Capture  of  Efficiencies  from  Design-to-Manufacture  Initiatives  and 
the  Planned  Test  Program.  Streamlined  design-to-manufacturing  initiatives  were  not 
all  realized.  Lor  example,  Lockheed  Martin  intended  to  use  a  new  unitized  wing  that 
would  reduce  production  costs  by  reducing  labor  required,  but  in  light  of  the  weight 
growth,  the  wing  had  to  be  redesigned  and  produced  “the  old  fashioned  way.”  Lock¬ 
heed  Martin  and  its  partners  used  different  versions  of  their  design  tools,  complicating 
their  ability  to  rapidly  integrate  designs.29  The  early  weight  growth  of  the  aircraft  may 
have  been  a  symptom  of  both  the  program’s  integration  complexities  and  the  difficulty 


27  “CAIG  Milestone  B  Independent  Cost  Estimate  (ICE)  for  Joint  Strike  Fighter  (JSF)  Program,”  2001. 

28  Telephone  conversation  with  JPO  representative,  May  11,  2010. 

29  Discussion  with  Lockheed  Martin,  April  23,  2010. 
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in  managing  or  controlling  design  elements  without  a  truly  common  system.  Lock¬ 
heed  Martin  had  planned  a  five-month  assembly  plan  through  the  use  of  design-for- 
manufacturing  assembly  methods,  advanced  manufacturing  processes,  and  assembly 
simulation  processes.  Lockheed  Martin  may  yet  reach  this  target;  thus  far,  it  takes 
about  30  months  to  produce  one  aircraft.30  Design  for  manufacturing  was  rated  a 
medium  risk  in  the  CARD  of  2000. 31 

The  baseline  estimate  for  Milestone  B  was  based  on  anticipated  savings  through 
streamlined  structural  and  flight  testing  with  fewer  ground  test  articles — four  rather 
than  seven — and  leaner  test  operations.32  Lockheed  Martin  claims  that  many  of  these 
design  issues  are  behind  them.  However,  the  test  program  is  still  in  its  early  stages,  and 
additional  design  refinements  may  be  necessary.  Lockheed  Martin’s  estimate  to  com¬ 
plete  the  test  program  has  increased  by  50  percent,  and  DoD’s  CAPE  is  looking  for 
ways  to  add  flight  test  articles  and  flying  hours  to  the  program.33  It  is  not  clear  whether 
the  move  to  increase  the  test  program  is  driven  by  experience  or  by  lack  of  confidence 
(or  risk  avoidance)  on  the  part  of  the  government,  which  may  feel  that  it  cannot  afford 
a  failure  (crash)  at  this  point  in  the  program. 

Weight  Growth,  Design  Changes,  and  Schedule  Slippage  Had  Cascading  Effects 
on  Other  Program  Costs.  The  problems  associated  with  the  weight  growth  and  air¬ 
craft  redesign  effort  were  not  entirely  cleared  up  by  the  2004  re-baselining  or  even  by 
the  2007  re-baselining.  The  changes  in  manufacturing  described  above  led  to  delays 
in  producing  definitive  design  specifications  for  second-  and  third-tier  suppliers  and 
changes  to  these  designs,  delays  in  the  test  program,  and  increases  in  the  cost  of  parts 
and  labor  because  of  changes  in  plans  and  the  stretch  in  the  schedule. 

These  execution  problems  drove  increases  not  just  in  RDT&E  costs  but  also  in 
procurement.  The  JSF  acquisition  strategy  depended  on  a  quick  ramp-up  to  high  pro¬ 
duction  rates  to  achieve  affordability.  Schedule  delays  have  led  to  a  slower  ramp-up, 
lower  rates  of  production,  and  an  extension  of  production  ten  years  beyond  that  of  the 
Milestone  B  program.  Figure  4.9  shows  the  implications  of  the  shift  in  production, 
with  the  red  and  green  bars  showing  the  2001  SAR  program  and  the  red  and  purple 
bars  showing  the  December  2009  SAR  program.  Procurement  costs,  even  in  constant 
dollars,  rise  with  the  stretching  out  of  production  costs.  In  addition  cost  increases  also 
result  from  differences  in  actual  and  estimated  labor  rates.  Actual  rates  have  been  in 
the  4-5  percent  per  year  range  whereas  the  DoD  rates  used  to  estimate  the  costs  of  the 
program  are  at  2.1  percent. 


30  Discussion  with  Lockheed  Martin,  April  23,  2010. 

31  F-35  Joint  Program  Office,  Joint  Strike  Fighter  Cost  Analysis  Requirements  Description ,  July  2001,  p.  11-8. 

32  Presentations  to  PARCA  team  by  Lockheed  Martin,  April  23,  2010. 

33  Presentations  to  PARCA  team  by  Lockheed  Martin,  April  23,  2010. 
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Figure  4.9 

Comparison  of  2001  SAR  Plan  to  2009  SAR  Plan 


Fiscal  year 

NOTE:  409  aircraft  were  canceled  between  2001  and  2002  and  therefore  do  not  appear  in  the  2009  SAR. 
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Poor  Performance  of  Government  or  Contract  Personnel.  Some  of  the  systems 
used  in  earlier  phases  of  the  program,  or  intended  to  be  used  during  SDD  as  a  way  to 
manage  or  oversee  the  effort,  seem  not  to  have  been  implemented  fully  in  the  SDD 
ph  ase.  The  use  of  CAIV  processes  and  implementation  of  an  EVM  system  both  seem 
to  fall  into  this  category.  Although  we  do  not  have  good  visibility  into  how  cost-design 
trades  were  made  after  MS  B,  interviews  with  participants  as  well  as  the  program’s  dif¬ 
ficulties  indicate  that  the  process  that  proved  useful  before  Milestone  B  was  not  effec¬ 
tive  afterward.  EVM  issues  were  studied  by  IDA  and  thus  are  not  addressed  here. 

Formal  CAIV  Dropped  After  SDD  Contract  Award.  Some  of  these  acquisition 
reform  initiatives,  such  as  the  use  of  cost/operational  performance  trades  (COPT) 
(implementation  of  CAIV),  were  extensively  employed  in  the  development  of  require¬ 
ments  and  seen  as  successful.  However,  these  efforts  were  not  continued  during  the 
competition  or  after  contract  award,  when  extensive  additional  design  decisions  were 
made.  In  fact,  a  discussion  with  requirements  officials  in  the  JPO  suggested  that  about 
two-thirds  of  the  aircraft  design  decisions  were  made  post-MS  B  and  were  not  included 
in  the  contract.  That  is,  in  the  name  of  acquisition  reform  and  in  an  effort  to  reduce 
redundant  government  oversight,  the  government  lost  a  voice  in  key  design  decisions 
where  trades  were  made.  Contract  incentives  were  put  in  place  to  guide  Lockheed 
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Martin,  but,  in  hindsight,  JPO  officials  do  not  think  these  achieved  the  desired  stra¬ 
tegic  goals.34 

Complicating  the  use  of  CAIV/COPT  was  the  services’  continuing  alteration  of 
the  design  requirements.  These  changes  did  not  affect  the  key  performance  parameters 
(KPP)  and  therefore  did  not  raise  any  red  flags.  Each  service  had  the  ability  to  request 
design  changes,  which,  because  of  the  goal  of  commonality,  affected  all  three  variants 
of  the  aircraft.  It  seems  that  the  design  changes  were  made  without  trade  studies  to 
justify  the  changes.35 


Conclusions 

Figure  4.10  traces  the  immediate  causes  of  cost  growth  to  intermediate  causes  and 
finally,  on  the  far  right,  root  causes.  Root  causes  are  those  factors  that,  if  not  present 
(or  absent),  might  have  resulted  in  a  different  outcome. 

The  immediate  cause  of  the  breach  is  APUC  and  PAUC  cost  growth  of  over 
50  percent  in  the  most  recent  government  estimate.  The  government’s  Nunn-McCurdy 


Figure  4.10 

Root  Cause  Analysis  Flowchart 
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34  Telephone  conversation  with  JPO  representative.  May  11,  2010. 

35  A  good  example  of  this  involves  the  evolution  of  the  internal  weapons  bay  storage.  Following  the  initial 
designs,  the  CV  variant’s  weapons  bay  was  increased  in  size  to  allow  it  to  carry  2,000  lb  bombs.  The  STOVL  and 
CTOL  variants  had  weapons  bays  only  large  enough  to  carry  1,000  lb  bombs,  but  for  commonality,  all  three  were 
redesigned  with  larger  weapons  bays.  This  redesign  caused  the  STOVL  variant  weight  and  balance  issues  and  was 
eventually  redesigned  again  with  the  smaller  weapons  bay. 


58  Root  Cause  Analyses  of  Nunn-McCurdy  Breaches,  Volume  1 


estimate  was  driven  by  a  program  history  of  schedule  slippage  and  substantial  cost 
growth  in  the  RDT&E  account.  This  resulted  in  government  estimates  of  future  pro¬ 
curement  cost  growth.  The  path  to  the  government’s  estimate  is  understandable  given 
the  program’s  history,  and  the  root  cause  discussion  revolves  around  how  this  history 
might  have  been  more  successful. 

The  root  cause  lies  in  some  measure  in  an  overly  optimistic  baseline  estimate  of 
the  influence  of  acquisition  reform  and  produceability  initiatives.  Though  difficult  to 
quantify,  belief  in  these  initiatives  combined  with  the  imperative  to  produce  a  replace¬ 
ment  fighter  for  the  F-16  enabled  OSD  to  undertake  a  technologically  complex,  highly 
concurrent  F-35  program.  Whether  a  prototype  would  have  helped  keep  the  program 
on  track  will  never  be  known,  but  the  program’s  experience  suggests  that  the  concept 
demonstrator  was  not  a  sufficient  basis  for  implementing  this  program.  The  lack  of 
time  and  incomplete  implementation  of  processes  to  refine  design  elements,  insufficient 
design  and  systems  integration  control,  and  the  focus  on  CTOL  rather  than  STOYL 
led  to  significant  problems  in  execution,  requiring  that  the  program  essentially  start 
over  two  years  in.  These  problems  in  design  and  manufacturing  execution — not  fund¬ 
ing  instability,  changes  in  quantity,  or  unreasonable  performance  expectations — led  to 
the  schedule  slippage  and  cost  growth  that  resulted  in  a  Nunn-McCurdy  breach. 

Lessons  from  this  program  that  might  help  to  avert  similar  situations  in  the  future 
include  the  important  warning  that  technology  demonstration  does  not  equal  pro¬ 
duceability.  If  produceability  is  a  concern  for  a  program,  and  it  often  is,  then  additional 
time,  trade  studies,  and  negotiation  are  likely  to  be  useful  early  in  the  SDD  phase. 
Parametric  estimates  for  fighters  are  based  largely  on  weight,  but  as  components  have 
become  smaller  and  integration  has  taken  a  prominent  place  in  fighter  production,  it 
may  be  important  to  consider  other  metrics  as  drivers  of  cost.  Furthermore,  although 
acquisition  reform  precepts  are  indeed  likely  to  result  in  greater  cost  savings  over  tra¬ 
ditional  approaches,  such  savings  may  not  accrue  in  the  first  few  programs  as  there 
will  be  a  cost  improvement  curve  in  the  implementation  and  application  of  such  tech¬ 
niques  as  CAIV,  design-to-production,  and  incentive  contracts.  Given  the  technical 
complexity  of  the  JSF  and  the  program  complexity  with  so  many  subcontractors,  the 
savings  through  implementation  of  design-to-production  or  other  such  manufacturing 
process  techniques  may  not  be  as  scalable  as  believed  by  OSD.  Those  who  watch  the 
auto  industry,  for  example,  note  that  the  JSF  is  orders  of  magnitude  more  complex  than 
designing  and  building  a  new  car.  This  raises  another  point  not  captured  in  the  cost 
estimates,  and  that  is  risk.  It  is  true  that  the  CAIG  built  in  an  allowance  for  6  percent 
weight  growth,  but  that  was  not  enough.  For  a  program  that  all  acknowledged  was 
risky,  there  were  not  enough  hedges  against  risk  or  explicit  incremental  checkpoints. 

Both  the  JPO  and  Lockheed  Martin  claim  to  have  been  surprised  at  the  lack  of 
an  SDD  manufacturing  planning  phase  during  which  there  might  have  been  more  col¬ 
laboration  on  design  specifications  and  discussion  of  the  trade-offs  to  be  made  between 
cost  and  commonality  and  performance.  Although  none  of  the  KPPs  changed  signifi- 
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candy  during  the  execution  of  SDD,  this  dog  that  didn’t  bark  should  perhaps  have 
been  a  signal  that  not  enough  collaboration  was  going  on.  As  retold  by  the  JPO,  Lock¬ 
heed  Martin  made  a  substantial  number  of  important  design  decisions  (as  much  as 
two-thirds  of  the  design  decisions  by  one  estimate)  on  their  own  with  only  informal 
government  participation  and  with  no  contractual  record  holding  Lockheed  Martin  to 
those  specifications.  This  devolution  of  responsibility  to  Lockheed  Martin  is  a  logical 
outgrowth  of  the  tenet  of  acquisition  reform  that  wanted  to  remove  the  government 
from  its  role  as  micro-manager,  but  in  this  case  the  JPO  might  say  that  the  pendulum 
swung  too  far  in  the  other  direction. 

Finally,  what  seems  to  have  been  inadequate  control  of  systems  integration 
allowed  such  substantial  weight  growth  that  the  aircraft  had  to  be  substantially  rede¬ 
signed.  Imperfect  implementation  of  CAIV  after  Milestone  B  and  a  lack  of  trade  stud¬ 
ies  also  contributed  to  weight  growth.  Even  after  the  major  elements  of  the  design  were 
finalized,  there  were  ongoing  changes  to  subcomponent  designs  that  led  to  delays  in 
production  and  testing.  Increases  in  rates  for  labor,  materials,  and  other  factors  were 
compounded  by  schedule  slippage.  Whether  the  estimates  that  pushed  the  program 
into  the  Nunn-McCurdy  breach  will  be  borne  out  remains  to  be  seen. 


Next  Steps 

Significant  cost  and  schedule  uncertainties  remain  in  the  JSF  program.  Lockheed 
Martin  claims  that  its  actual  costs  since  2006  show  very  limited  cost  growth  and 
downward  curves  are  anticipated.  However,  much  technical  and  production  uncer¬ 
tainty  remains  in  the  program.  Flight  test  is  still  in  the  early  stages  and  has  relied 
heavily  on  simulation.  Thus  far,  this  approach  seems  to  be  successful  although  there 
has  been  an  increase  in  the  use  of  traditional  testing  approaches.  Regardless  of  the  test 
method,  findings  from  these  tests  will  have  to  be  integrated  into  the  program.  Mission 
systems  have  yet  to  be  produced,  integrated,  and  tested,  and  many  account  these  to  be 
more  complex  than  the  airframe  tasks  completed  thus  far.  As  noted  above,  the  com¬ 
plete  effect  on  cost  of  reductions  in  commonality  between  the  variants  remains  to  be 
seen  because  they  will  affect  production  and  O&S  costs  into  the  future.  The  company 
states  that  international  suppliers  will  not  affect  program  schedule,  but  there  may  be 
added  risk  as  some  new  suppliers  are  brought  into  the  program,  and  certainly  there  will 
be  added  integration  complexity. 

If  the  CAPE-revised  estimates  are  indeed  only  through  2015,  then  even  the  impact 
of  inflation  rates  alone  will  result  in  significant  increases  to  program  costs.  These  infla¬ 
tion  factors  will  continue  to  contribute  to  cost  growth  until  the  program  moves  to  a 
fixed-price  contract  where  the  contractor  must  absorb  this  risk. 


CHAPTER  FIVE 


Longbow  Apache  Helicopter  (Apache  Block  III) 


This  chapter  discusses  the  program  to  upgrade  all  of  the  AH-64  Apache  attack  heli¬ 
copters  to  the  Apache  Longbow  Block  III  (AB3)  configuration  and  add  additional  AB3 
helicopters.  It  begins  with  a  brief  overview  of  the  program,  our  research  approach  to 
the  issue,  and  a  discussion  of  the  nature  of  the  Nunn-McCurdy  breach. 


Program  Overview 

The  AH-64  Apache  Longbow  is  an  armored  attack  helicopter  designed  to  attack 
enemy  tanks  at  extended  ranges.  Its  weapons  currently  include  Hellfire  antitank  mis¬ 
siles,  a  30-mm  chain  gun,  and  free-flight  rockets.  It  can  operate  at  night,  in  adverse 
weather,  and  in  the  presence  of  obscurities  such  as  smoke.  The  Longbow  radar  gives  the 
aircraft  higher  lethality,  better  survivability,  and  reduced  maintenance  requirements. 
The  program  involves  retrofitting  the  current  Apache  fleet  with  the  Longbow  Block  III 
upgrades  and  buying  56  new  AB3  aircraft.  The  Block  II  improvements  add  several 
technologies  to  the  aircraft,  including  digital  and  communications  improvements.1 
Many  of  these  technologies  were  added  in  anticipation  of  the  helicopter  operating  in 
conjunction  with  the  Future  Combat  System,  the  equipment  component  of  which  has 
been  canceled. 


Research  Approach 

Our  research  effort  focused  on  the  following  elements: 

•  understanding  the  evolution  of  the  AB3  program  from  Milestone  B  to  the  present 

•  understanding  the  differences  in  the  costs  of  (1)  the  48  new  build  aircraft  for  the 
13th  Combat  Aviation  Brigade  (CAB),  (2)  the  eight  new  build  aircraft  for  the 
training  base,  and  (3)  over  50  new  build  aircraft  manufactured  to  replace  Apache 


1  “AH-64D  Longbow  Block  II/Extended  Block  II,”  GlobalSecurity.org,  undated. 
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helicopters  lost  in  combat  in  Operation  Enduring  Freedom  (OEF)  and  Operation 
Iraqi  Freedom  (OIF) 

•  obtaining  a  breakdown  and  understanding  of  the  other  program  costs  reflected  in 
the  cost  increases  cited  above. 

The  initial  steps  in  this  effort  focused  on  obtaining  the  SARs  and  relevant  POMs 
and  other  official  documents  that  could  provide  details  about  the  various  increases  in 
program  cost  and  associated  explanations.  These  materials  provided  an  initial  set  of 
cost  numbers  but  failed  to  provide  cost  breakdowns  at  a  level  of  specificity  that  permit¬ 
ted  the  kind  of  assessment  that  was  being  sought  by  means  of  the  seven  statutory  root 
cause  factors  reflected  in  the  PARCA  root  cause  narrative  matrix  (see  below).  To  rectify 
this  shortfall  in  information,  RAND,  in  concert  with  the  PARCA  AB3  assessment 
leader,  contacted  the  AB3  Program  Office  and  initiated  a  sequence  of  exchanges  that 
clarified  the  cost  increases  of  interest  (as  described  below).  Of  particular  interest  in  this 
regard  was  the  increase  in  costs  associated  with  the  multiple  technology  improvements 
that  constituted  the  principal  rationale  for  the  AB3  program. 


Nunn-McCurdy  Breach 

This  section  provides  the  details  for  the  “speeding  ticket”  for  the  AB3  program — as 
summarized  in  Table  5.1.  The  AB3  breach  was  triggered  by  the  31  percent  increase  in 
the  APUC  and  the  nearly  26  percent  increase  in  the  PAUC. 

The  major  contributors  to  the  unit  cost  growth  that  resulted  in  the  AB3  program 
Nunn-McCurdy  breaches  summarized  in  Table  5.1  were  as  follows: 

The  December  31,  2006,  SAR  increased  the  number  of  AB3  remanufactured 
aircraft  by  37 — from  597  to  634 — and  stretched  out  the  annual  procurement  for 
FY  2010-2024  by  shifting  more  of  the  production  to  later  years.  This  increased  pro¬ 
gram  cost  by  $578.7  million  (+10.5  percent).  Software  maintenance  and  system  engi¬ 
neering/program  management  costs  derivative  of  the  increase  in  aircraft  quantity  and 
stretch-out  of  the  procurement  also  contributed  to  this  cost  increase.  The  June  2007 
APB  reflected  these  program  changes  and  established  a  new  (current)  baseline,  which 
reflected  in  the  December  31,  2007,  SAR. 

The  final  version  of  the  December  31,  2008,  SAR  reflected  a  decision  to  buy  new 
airframes  (competitive  in  cost  and  increasing  projected  airframe  life  compared  with 
refurbishing  older/original  Apache  airframes)  and  also  included  an  increase  in  nonre¬ 
curring  engineering  (NRE)  costs  reflecting  experience  and  information  gained  in  the 
period  after  Milestone  B.  These  changes  added  $89.0  million  (BY  $)  and  $136.0  mil¬ 
lion  (BY  $),  respectively,  to  the  program  cost.2  The  NRE  costs  included  inter  alia 
retooling  costs  unanticipated  at  Milestone  B  and  a  decision  to  include  existing  older 


2  Information  provided  by  the  AB3  Program  Office,  May  10,  2010. 


Table  5.1 

AB3  Nunn-McCurdy  Breach  Summary  (Costs  in  FY  2006  $  Millions) 


Cost  Growth  Threshold  Breaches 


Baseline 

Current  Estimate 

Quantity 

Unit 

(December  2009 

Baseline 

Baseline 

(December 

Cause  in 

Cost 

SAR) 

Breached 

Percentage 

Amount 

Level 

Quantity 

2009  SAR) 

SAR 

SAR  Explanation 

APUC  9.600 

APUC, 

Over  current 

APUC 

+  $2,991 

Critical 

639 

695 

Quantity 

56  new  build 

2007  APB 

$12,591 

baseline 

+31.16 

change  from 

aircraft 

(2007  APB) 

remanufacture 

significantly 

PAUC  11.139 

PAUC, 

PAUC 

+  $2,838 

Critical 

634 

690 

of  634  aircraft 

more  expensive 

2007  APB 

$13,977 

+25.48 

to  building 
additional  56 

to  build  because 
they  are  100% 

APUC  9.225 

APUC, 

Over  original 

APUC 

+  $3,366 

Significant 

597 

690 

new  aircraft 

new  versus 

2006  APB 

$12,591 

baseline 

+36.49 

30%  new  for  a 

(2006  APB)  remanufacture 

SOURCE:  Department  of  Defense,  Selected  Acquisition  Report  AB3,  December  31,  2009. 

NOTE:  The  numbers  in  red  indicate  the  "speeding  ticket"  triffering  root  cause  analysis  by  PARCA. 
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software  in  the  51  LRIP  aircraft  when  ir  became  clear  that  some  of  the  AB3  software 
anticipated  to  be  included  in  LRIP  aircraft  would  not  be  available. 

In  August  2009,  the  AB3  Program  Office3  estimated  that  there  would  be  addi¬ 
tional  recurring  procurement  material  costs  associated  with  (1)  increased  cost  of  the 
improved  drive  system  ($97.2  million,  BY  $),  (2)  increased  cost  of  the  composite  main 
rotor  blade  ($21.4  million,  BY  $),  and  (3)  a  range  of  LRIP  enhancements  ($86.4  million, 
BY  $).  The  last  of  these  increases  was  due  primarily  to  a  (previously  anticipated)  deci¬ 
sion  to  equip  the  AB3  with  high  performance  shock  struts  ($80.7  million,  BY  $).  This 
addition  is  a  direct  consequence  of  the  growth  in  aircraft  weight  from  the  Apache  A  to 
the  Apache  D  as  a  result  of  the  weight  that  went  along  with  enhanced  mission  pack¬ 
ages.  This  generated  a  perceived  need  to  reestablish  prior  standards  for  the  AB3  to 
allow  aircraft  landings  under  difficult  conditions  with  no  damage  to  the  aircraft. 

The  December  31,  2009,  SAR  reflected  the  above  cost  increases  and  also  included 
costs  associated  with  directed  changes  to  the  program  (RMD  700  and  RMD  802), 
which  added,  respectively,  48  new  build  AB3  for  a  13th  CAB  ($1,925  million,  BY  $) 
and  eight  new  build  AB3  for  the  training  base  ($225.9  million,  BY  $).  This  SAR  also 
reflected  increases  in  labor  rates  ($18.2  million,  BY  $).  These  changes  and  those  noted 
in  the  August  2009  POM  increased  the  total  program  cost  by  $2,597.3  million  (BY  $) 
and  the  procurement  cost  by  $2,602.0  million  (BY  $)  from  the  current  (June  2007) 
APB  baseline.  As  noted  in  Table  5.1,  the  additional  56  new  build  aircraft  were  the  prin¬ 
cipal  source  of  the  cost  increases  that  led  to  the  Nunn-McCurdy  breach. 

After  the  closing  date  for  input  to  the  December  31,  2009,  SAR  (in  preparation 
for  Milestone  C  and  in  part  a  consequence  of  the  Nunn-McCurdy  breach),  reexami¬ 
nation  of  cost  estimates  for  many  of  the  technology  improvements  to  be  incorporated 
in  post-LRIP  production  has  raised  major  questions  about  the  validity  of  the  original 
AB3  cost  estimates  for  these  improvements.  As  a  consequence,  it  is  clear  that  a  new 
set  of  AB3  cost  estimates  that  increase  the  costs  of  the  program  beyond  those  sum¬ 
marized  above  and  shown  in  Table  5.1  will  shortly  emerge.  The  sources  for  these  addi¬ 
tional  costs  will  include  (1)  increases  in  RDT&E  costs  as  technical  improvements  are 
brought  to  the  stage  where  they  meet  operational  requirements  and  (2)  more  realistic 
cost  improvement  curves  associated  with  the  large  amount  of  touch  labor  (that  is, 
hands-on  labor)  required  in  the  remanufacturing  process. 


Findings  Narrative 

Table  5.2  provides  the  AB3  version  of  the  PARCA  root  cause  narrative  matrix,  reflect¬ 
ing  the  program  changes  described  above.  The  cell  entries  correspond  to  the  fiscal  year 
in  which  the  changes  were  made. 


3  Information  provided  by  the  AB3  Program  Office,  May  10,  2010. 
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Table  5.2 

PARCA  Root  Cause  Narrative  Matrix  for  the  AB3 


Year  from  MS  B  and  Fiscal  Year 

B 

2006 

1 

2007 

2 

2008 

3 

2009 

4 

2010 

5 

2011 

Baseline  issues 

Unrealistic  estimates 
for  cost  or  schedule 

Technology 
development 
and  cost 
improvement 
curve  issues 

X 

X 

X 

X 

X 

Immature 

technology;  excessive 
manufacturing, 
integration  risk 

X 

X 

X 

X 

X 

Unrealistic 

performance 

expectations 

X 

X 

X 

X 

X 

Execution  issues 

Changes  in 

procurement  quantity 

X 

Upgrade 
all  aircraft; 

stretch 

production 

New  build  of 

56  aircraft 

Inadequate  funding/ 
funding  instability 

X 

Unanticipated 
design,  engineering, 
manufacturing,  or 
technical  issues 

X 

Buy  new 
airframes; 
NRE  costs 

Poor  performance 
of  government  or 
contract  personnel 

X 

Other 

X 

Observations  on  each  of  the  cell  entries  follow  (in  chronological  order). 

Unrealistic  Cost  Estimate  (Milestone  B:  Technology  Development  Issues).  Any 

program  that  incorporates  a  large  number  of  cutting-edge  technologies  such  as  the 
AB3  must  of  necessity  depend  on  past  technology  development  programs  to  estimate 
the  eventual  costs  of  a  technology  developed  for  military  application — an  inherently 
challenging  undertaking  considering  military  performance  standards.  When  a  new 
program  incorporates  as  many  technologies  as  the  AB3  (15  by  Army  accounts;  more 
by  others),  this  problem  is  magnified,  and  the  statistical  probability  that  a  number  of 
these  technologies  will  manifest  unanticipated  development  problems  correspondingly 
increases.  In  circumstances  such  as  this,  program  proponents  tend  to  be  optimistic 
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about  eventual  technology  development  and  insertion  costs,  and  thus  DoD  acquisi¬ 
tion  management  must  take  a  cautious  if  not  a  contrary  approach.  The  AB3  program 
appears  to  manifest  such  a  situation  as  already  reflected  in  the  increase  in  estimated 
cost  of  the  improved  drive  system  ($97.2  million,  BY  $)  on  a  Milestone  B  base  cost  esti¬ 
mate  of  $235.4  million  (BY  $) — a  41  percent  increase  on  a  major  technology  insertion. 
In  comparison  (and  somewhat  in  contrast),  the  increase  in  cost  of  the  composite  main 
rotor  blade  is  estimated  at  $21.4  million  (BY  $)  on  a  base  of  $204.0  million  (BY  $) — a 
10  percent  increase  over  original  estimates.  (See  the  final  section  of  this  chapter  for  a 
further  discussion  of  the  cost  estimation  challenges  for  these  technology  insertions.)  A 
further  broad  overall  complication  in  the  costing  for  the  AB3  occurred  at  Milestone  B 
in  June  2006  when  the  Army  Cost  Estimate  (ACE)  was  chosen  as  the  program  baseline 
in  spite  of  a  significantly  higher  estimate  by  the  OSD  CAIG. 

Unrealistic  Cost  Estimate  (Milestone  B:  Cost  Improvement  Curve  Issues).  As  with 
the  eventual  costs  of  a  cutting-edge  technology  military  system,  substantial  uncertain¬ 
ties  also  surround  the  actual  costs  of  incorporating  such  systems  along  with  many 
others  in  the  actual  manufacturing  process.  Here,  there  is  the  challenge  of  estimating 
just  how  fast  those  technicians  who  must  perform  the  actual  insertion  of  the  finished 
product  into  a  complex  military  system  such  as  a  heavily  armed  helicopter  equipped 
with  multiple  sensors  can  achieve  an  efficient  production  process.  In  this  case,  it  is 
already  clear  that  the  manufacturing  costs  for  the  LRIP  aircraft  will  be  substantially 
greater  than  originally  anticipated.  In  these  circumstances,  it  can  be  expected  that 
there  will  be  a  strong  temptation  to  be  optimistic  about  the  so-called  cost  improve¬ 
ment  curve.  As  with  development  and  procurement  costs,  in  these  circumstances  DoD 
acquisition  management  must  take  a  skeptical  view  of  such  optimism.  This  situation  is 
manifest  in  the  AB3  where  the  downstream  make-up  in  the  cost  improvement  curve 
for  the  remanufacturing  process  (see  Figure  5.1)  appears  to  some  experienced  observ¬ 
ers  as  unduly  optimistic,  although  actual  experience  in  this  regard  remains  to  be  seen. 

Changes  in  Procurement  Quantity 

Year  1:  Upgrade  All  Aircraft;  Stretch  Production.  The  decision  in  the  December 
2006  SAR  to  increase  the  number  of  AB3  remanufactured  aircraft  by  37  aircraft  from 
597  to  634  and  to  stretch  the  annual  procurement  buy  profile  for  FY  2010-2024  fol¬ 
lowed  on  the  heels  of  Milestone  B.  In  consideration  of  the  timing,  the  decision  appears 
to  manifest  an  early  recognition  that  the  Milestone  B  assessment  was  premature,  and 
there  needs  to  be  a  more  realistic  baseline  program  as  reflected  in  the  June  2007  DAB 
decision  to  establish  a  new  APB  for  the  program. 

Year  4:  Add  56  New  Build  Aircraft.  Based  on  available  information,  it  appears 
that  the  decision  to  add  56  new  build  aircraft  to  the  AB3  program  instead  of  creating  a 
new  line  item  for  these  aircraft  was  in  part  a  decision  made  in  consultation  with  DoD 
legal  and  comptroller  representatives.  This  decision  alone,  independent  of  other  pro¬ 
gram  cost  increases,  would  by  inspection  (i.e.,  back-of-the-envelope  calculations)  have 
assuredly  produced  a  Nunn-McCurdy  breach  at  the  time  of  the  next  program  SAR. 
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Figure  5.1 

Meeting  2009  Production  Estimate  Requires  Realization  of  Greater  Learning 
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Although  the  discussion  above  indicates  that  there  was  about  to  be  unanticipated  cost 
growth  in  the  program,  this  decision  alone  could  be  characterized  as  the  flagship  root 
cause  for  the  breach. 

Unanticipated  Manufacturing  Issues 

Year  3:  Buy  New  Airframes.  Apparently  as  a  result  of  unanticipated  levels  of 
damage  to  aircraft  returning  from  OEF  and  OIF  and  going  through  the  Apache 
Block  II  upgrade  program,  it  became  clear  that  it  was  more  cost  effective  in  the  long 
run  (e.g.,  projected  aircraft  longevity)  to  purchase  new  airframes  rather  than  repair 
existing  ones. 

Year  3:  Acknowledge  NRE  Costs.  As  discussed  above,  the  challenge  of  estimating 
development  and  insertion  costs  for  cutting-edge  technologies  is  well  recognized,  as  is 
the  tendency  for  system  proponents  to  underestimate  such  costs. 


AB3  Technology  Issues 

As  emphasized  in  the  above  assessment  of  the  contributors  to  the  AB3  Nunn-McCurdy 
breach,  it  is  clearly  important  to  understand  those  increased  costs  associated  with 
major  technology  issues,  in  particular  the  three  major  technology  contributions  to  the 
relevant  cost  increases: 


68  Root  Cause  Analyses  of  Nunn-McCurdy  Breaches,  Volume  1 


•  improved  drive  system 

•  composite  main  rotor  blade 

•  high  performance  shock  struts  (HPSS). 


Acquisition  Issues 

As  argued  above,  a  number  of  lessons  can  be  derived  from  the  AB3  experience  that  are 
relevant  to  the  acquisition  process: 

•  Using  the  program  manager’s  cost  estimate  in  the  SARs  introduces  bias  into  the 
system;  CAPE  should  carefully  review  the  SAR  inputs. 

•  Cutting-edge  technology  development  and  insertion  costs  are  harder  to  predict; 
there  is  a  high  likelihood  that  unit  cost  will  grow  as  a  function  of  the  number  of 
advanced  subsystems  being  incorporated  (i.e.,  at  least  one  subsystem  out  of  a  large 
batch  of  unproven  subsystems  is  bound  to  present  a  problem). 

•  Combining  remanufactured  and  new  build  items  in  the  same  program  will 
increase  the  risk  of  cost  growth. 

•  The  balance  between  capability  requirements  and  payload  is  very  delicate. 

•  When  there  are  many  technical  insertions,  some  of  which  depend  on  others,  pre¬ 
dicting  the  cumulative  effect  on  program  timing  becomes  very  difficult. 

•  If  significant  engineering  design  changes  made  after  the  baseline  unit  cost  at 
Milestone  B  have  been  estimated  (e.g.,  in  the  AB3  case  replacing  the  airframe 
with  a  new  one),  then  the  unit  cost  will  surely  increase — possibly  implying  a  need 
to  include  contingency  dollars  to  cover  such  after-the-fact  design  changes. 


AB3  Technology  Assessment 

Fifteen  key  technologies  associated  with  the  Apache  Longbow  Block  III  (AB3)  air¬ 
craft  were  initially  identified  as  potential  critical  technologies.4  These  15  technologies 
are  listed  in  Table  5.3  along  with  the  Army’s  assessment  of  their  TRLs  for  two  time 
periods — April  2004  and  March  2006.  (See  Appendix  C  for  a  definition  of  the  nine 
TRL  levels.) 

Army  analysis  subsequently  concluded  that  only  one  was  indeed  a  critical 
technology — the  improved  drive  system;  the  remaining  technologies  were  not  criti¬ 
cal.  Note  that  the  TRL  rating  for  the  drive  system  improvements  was  designated  as 
level  6  in  both  years.  (TRL  6  means  that  a  representative  model  or  prototype  system 


4  U.S.  Government  Accountability  Office,  Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs,  GAO- 
07-406SP,  March  2007,  p.  106. 
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Table  5.3 

Assessment  of  AB3  Technologies 


Technology  Insertions 

TRL 

April  2004 

TRL 

March  2006 

1 

Open  system  architecture 

5 

6 

2 

Composite  main  rotor  blade  (CMRB) 

6 

7 

3 

Modernized  signal  processor  unit 

7 

7 

4 

Instrument  flight  rules/meteorological 

9 

9 

5 

Multimode  laser 

5 

7 

6 

Radio  frequency  interferometer  (RFI)  frequency  extension 

6 

7 

7 

Image  fusion 

5 

8 

8 

Aided  target  detection/classification 

5 

8 

9 

Modified  7700-GE-701D  engine 

8 

8 

10 

Level  4  unmanned  aerial  vehicle  control 

6 

6 

11 

Improved  drive  system 

6 

6 

12 

Fire  control  radar  (FCR)  maritime  targeting  mode 

5 

6 

13 

FCR  range  extension 

6 

6 

14 

RFI  passive  ranging 

5 

6 

15 

Cognitive  decision  aiding  system 

6 

6 

that  is  well  beyond  the  breadboard  level  has  been  tested  in  a  relevant,  nonoperational 
environment.) 

Other  potential  technology  insertions  that  could  result  in  developmental  delays, 
cost  increases,  or  both  are  (1)  the  modified  T700-GE-701D  engine,  (2)  the  CMRB, 
and  (3)  the  HPSS  advanced  landing  gear. 

The  701D  engine  was  given  a  high  TRL  rating  (8)  in  both  years.  The  CMRB, 
while  based  on  a  proven  experimental  design,  received  TRL  ratings  of  6  and  7  in  April 
2004  and  March  2006,  respectively.  Early  on,  concerns  had  been  expressed  that  the 
composite  blade  might  lead  to  a  cost  growth.  Finally,  the  decision  to  add  the  EIPSS 
advanced  landing  gear  was  apparently  an  afterthought.  It  was  not  considered  in  2006 
and  probably  would  not  have  been  on  the  advanced  technology  list  because  shock 
struts  are  a  mature  technology.  Elowever,  their  incorporation  into  the  LRIP  units  raises 
the  unit  cost  of  the  aircraft. 

Each  of  these  four  technologies  is  assessed  below  in  terms  of  its  potential  develop¬ 
mental  delays  or  cost  growth. 
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Drive  System  Improvements 

The  drive  system  improvements  consist  of  the  Rotorcraft  Drive  System  of  the  21st 
Century  (RDS-21)  face  gear  transmission.  The  RDS-21  with  its  split  torque  capabil¬ 
ity  allows  significantly  more  torque  to  be  transferred  to  the  rotor  by  combining  the 
output  torque  of  the  two  engines  into  a  single  power  torque  transmission.  This  is  to  be 
accomplished  without  any  increase  in  transmission  weight.  The  RDS-21  technology 
was  transferred  to  the  Apache  program  manager  in  2006.  It  has  completed  qualifica¬ 
tion  testing  and  is  now  slated  to  enter  LRIP  in  2011.  The  estimate  at  Milestone  B  was 
based  on  the  V-22  transmission  cost  (with  a  similar  power  rating)  plus  the  addition  of 
a  complexity  factor.  The  cost  growth  estimated  as  of  August  2009  by  the  Apache  Long¬ 
bow  Block  III  Program  Office  is  $97.2  million  (BY  $). 

701 D  Engine 

The  T700-GE-701D  engine  rating  of  TRL  8  in  March  2006  means  that,  at  that  time, 
the  actual  system  was  completed,  and  it  was  flight  qualified  through  test  and  demon¬ 
stration.  The  first  flight  of  a  701D-powered  Apache  Longbow  Block  III  was  success¬ 
fully  made  in  August  2008.  The  701D  engine,  rated  at  1,449  shaft  horsepower,5  fea¬ 
tures  improved  hot-section  components  that  increase  the  durability  and  provide  greater 
power  than  the  current  T700-  GE-701C  engine.6  The  empty  weight  of  the  Block  III  air¬ 
craft  has  increased  approximately  6  percent  over  the  Block  II  aircraft  to  12,530  lb,  and 
the  shaft  horsepower  generated  from  each  engine  has  increased  a  comparable  amount. 
The  performance  specification  of  the  Apache  Longbow  Block  II  aircraft  requires  that 
the  aircraft  hover  with  a  3,400  lb  payload  at  6000  ft  altitude  at  95°F  temperature.7  The 
701D  engine  will  also  be  used  to  upgrade  the  Army’s  fleet  of  UH-60A  and  UH-60L 
Blackhawk  helicopters  to  the  new  UH-60M  configuration.  As  a  result,  the  engines  are 
being  furnished  to  the  contractor  as  government-furnished  equipment. 

Composite  Main  Rotor  Blade 

With  more  aerodynamically  efficient  airfoil  shape  and  a  higher  overall  twist  rate,  the 
CMRB  was  designed  to  provide  additional  lift  and  thus  improve  aircraft  performance, 
some  of  which  was  lost  on  account  of  the  weight  differential  between  the  Apache  A 
and  D  models.8  In  2003,  Boeing  reported  that  the  CMRB  would  cost  25  percent  less 


5  Personal  communication  from  Paul  T.  Keil,  contractor,  Apache  Longbow  Block  III  Program  Office,  May  12, 
2010. 

6  The  703D  improvements  include  horse  power  improvement  of  5.5  percent  at  standard-day  sea  level;  hot  tem¬ 
perature  increase  of  70°C;  improved  thermal  cooling  in  the  hot  section  rotors  and  stators;  gas  generator  cooling 
holes  and  slots;  reduced  pressure  loss  internal  blade  channeling;  blade  aluminum  oxide  coating;  and  hot  section 
airfoil  redesign  to  reduce  dovetail  stress.  See  Apache  Program  Office,  AH-64D  Longbow  Apache  Block  III,  Draft 
Procurement  Objective  Cost  Analysis  Requirements  Description  (CARD),  February  2010. 

7  Personal  communication  with  Paul  T.  Keil,  May  12,  2010. 

8  Graham  Warwick,  “Team  Player,”  Aviation  Week  &  Space  Technology,  February  16,  2009. 
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than  the  current  blades  and  would  have  twice  the  operational  life,  implying  an  even 
greater  reduction  in  total  O&S  cost.9  In  March  2006,  the  CMRB  was  given  a  TRL 
rating  of  7,  indicating  that  a  prototype  had  been  sufficiently  demonstrated  in  an  opera¬ 
tional  environment.  However,  the  cost  estimate  methodologies  used  at  MS  B  for  the 
composite  blade  were  based  on  analogous  calculations  of  an  older  blade  (presumably 
not  composite?),  which  may  partially  explain  why  concerns  were  expressed  early  that 
the  CMRB  might  contribute  to  cost  growth.  The  Program  Office  now  knows  much 
more  about  the  blade  and  expects  that  the  increased  operational  lifetime  of  the  blades 
will  offset  at  least  partially  the  cost  growth  experienced  thus  far. 

High-Performance  Shock  Strut 

The  HPSS  advanced  landing  gear  was  scheduled  to  enter  LRIP  in  2010  and  to  pro¬ 
vide  the  aircraft  with  hard  landing  capabilities.  It  improves  on  the  struts  used  in  the 
Apache  Longbow  Block  II  aircraft,  which  were  facing  obsolescence  and  not  perform¬ 
ing  to  standard  in  the  field.  The  HPSS  was  also  developed  to  address  the  gross  weight 
increase  of  the  Apache  D  model,  for  which  the  old  shock  struts  were  no  longer  optimal. 
Consisting  of  a  new  valve  design  that  can  be  retrofitted  onto  the  aircraft,  this  technol¬ 
ogy  restores  the  helicopter’s  ability  to  meet  its  crash  performance  requirements.  Since 
the  new  shock  strut  was  not  included  in  the  MS  B  baseline,  its  incorporation  in  LRIP 
units  raises  the  unit  cost  of  the  aircraft. 


Conclusion 

The  key  contributor  to  the  cost  growth  in  the  AB3  program  was  the  decision  to  com¬ 
bine  58  new  build  aircraft  with  the  MDAP  baseline  remanufacturing  program  and 
include  them  in  the  PAUC/APUC  calculations  of  the  ongoing  AB3  remanufacturing 
program  rather  than  to  open  a  new  production  MDAP  reporting  line.  The  Army  has 
subsequently  separated  the  new  build  aircraft  from  the  remanufactured  ones  for  pur¬ 
poses  of  program  reporting.  Compounding  this  decision  were  cost  increases  engen¬ 
dered  by  the  improved  drive  system  and  the  decision  to  add  the  high-performance 
shock  strut,  which  was  not  in  the  MS  B  baseline.  Exacerbating  these  problems  were 
unrealistic  cost  and  schedule  estimates,  unrealistic  expectations  for  cost  improvements, 
and  inclusion  of  several  cutting-edge  technologies. 


9  The  Boeing  Company,  “Apache  Longbow  Flies  Toward  Future  with  New  Composite  Rotor  Blades,”  Decem¬ 
ber  3,  2003. 


CHAPTER  SIX 


Wideband  Global  Satellite 


This  chapter  presents  the  final  of  four  case  studies.  It  begins  by  briefly  describing  the 
program  and  then  the  specific  nature  of  the  Nunn-McCurdy  breach  that  occurred 
along  with  the  sources  of  that  breach.  It  then  presents  the  root  cause  analysis. 


Program  Overview 

To  meet  DoD’s  ever-growing  demand  for  military  satellite  communications  (SATCOM), 
the  WGS  program  was  funded  in  2001  (originally  known  as  the  Wideband  Gapfiller 
Satellite  program)  to  acquire  an  unprotected  wideband  SATCOM  capability  by  using 
a  commercial  off-the-shelf  satellite  bus  and  Ka-band  technology.  WGS  provides  both 
X-band  communications  compatible  with  the  older  Defense  Satellite  Communication 
System  (DSCS)  platforms  and  Ka-band  broadcast  capability  like  the  Global  Broadcast 
System  (GBS).  In  addition  to  DSCS  and  GBS  capabilities,  WGS  is  also  capable  of 
point-to-point  Ka-band  broadband  connections  and  has  a  cross-linked  communication 
bus  that  allows  signals  to  be  received  on  X-band  and  retransmitted  in  Ka-band  and 
vice  versa.  Throughput  for  each  satellite  is  estimated  at  over  two  gigabits  per  second.1 

The  program  consists  of  two  phases  or  “blocks.”  Block  I  of  WGS  comprises  three 
satellites,  the  last  of  which  was  placed  in  orbit  in  December  2009.  WGS  Block  II  con¬ 
sists  of  three  additional  satellites,  two  contracted  for  the  United  States  to  replace  aging 
DSCS  and  GBS  satellites  and  a  third  wholly  purchased  by  Australia  in  exchange  for 
a  percentage  of  global  WGS  bandwidth.  Block  II  satellites  are  essentially  the  same  as 
Block  I  with  a  high-bandwidth  bypass  feature  for  aerial  intelligence,  surveillance,  and 
reconnaissance  platforms.2  With  the  delays  and  eventual  cancellation  of  the  Transfor¬ 
mational  Satellite  Communications  System,  DoD  decided  to  procure  the  seventh  and 
eighth  WGS  satellites,  Block  II  Follow-on  (Ilf),  with  a  planned  total  buy  of  12  WGS 
satellites  to  meet  future  broadband  communication  requirements. 


1  U.S.  Air  Force,  The  Air  Force  Handbook,  Washington,  D.C.,  2007. 

2  “Block  1  of  WGS  Constellation  Completed,”  Aviation  Week  &  Space  Technology ,  March  8,  2010,  p.  16. 
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Nunn-McCurdy  Breach 

The  unit  cost  to  the  government  of  WGS  Block  II  was  roughly  50  percent  more 
expensive  than  Block  I  ($377  million  compared  with  $239  million),  and  Block  Ilf  is 
again  roughly  50  percent  more  expensive  than  Block  II  ($574  million  compared  with 
$377  million).  (See  Table  6.1.)  This  increase,  we  will  argue,  is  largely  due  to  the  stop¬ 
ping  and  restarting  of  the  production  line  and  the  fact  that  the  commercial  market 
no  longer  supports  WGS  systems,  which  have  not  changed  in  the  decade  since  initial 
design.  Such  increases  in  the  cost  to  the  government  resulted  in  a  Nunn-McCurdy 
breach  reported  to  Congress  in  March  2010. 

Table  6.1,  which  shows  the  APUC  in  constant  BY  2001  dollars,  illustrates  the 
Nunn-McCurdy  breach.  The  27  percent  increase  between  the  current  estimate  and  the 
current  APB  (third  column)  exceeds  the  25  percent  threshold  for  a  “critical”  breach 
(the  40  percent  increase  [fourth  column]  between  the  current  estimate  and  the  original 
APB  represents  a  “significant”  but  not  “critical”  Nunn-McCurdy  breach). 

The  averages,  in  turn,  permit  calculation  of  a  unit  cost  for  Block  I  (WGS  1-3), 
Block  II  (WGS  4  and  5),  and  Block  II  follow-on  (WGS  7  and  WSG  8) — but  not  in  a 
straightforward  manner.3  In  real  (BY  2001  $)  terms,  the  PAUC  of  the  WGS  satellite 


Table  6.1 

WGS  APUC  (Exclusive  of  Launch  Costs) 


Original  APB 

Current  APB/ 
Original  APB 

Estimate/ 
Current  APB 

Estimate/ 
Original  APB 

Block 

1 

1  &  II 

1,  II,  &  Ilf 

1,  II,  &  Ilf 

Satellites 

1-3 

1-5 

1-8a 

1-8a 

Contract  type 

FFP 

FPIF 

FPIF 

FPIF 

APUC,  $  millions 

268 

294 

374 

374 

Unit  costb,  $  millions 

239 

377c 

574 

574 

%  A  APUC,  % 

— 

110 

127 

140 

%  A  unit  cost,  % 

— 

158 

152 

240 

SOURCE:  Secretary  of  the  Air  Force  (SAF)  briefing  charts. 

NOTES:  APUC  costs  are  in  constant  BY  2001  dollars.  FFP  =  firm  fixed  price;  FPIF  =  fixed  price 
incentive  fee. 

a  WGS  6  was  purchased  for  Australia  and  does  not  show  up  in  U.S  budget  accounts. 
b  That  is,  cost  to  the  government. 

c  Cost  claims  currently  made  by  Boeing  suggest  that  the  true  cost  of  the  first  three  satellites 
was  roughly  $377  million.  We  discuss  this  in  Appendix  E. 


3  Note  that  the  original  APB  was  $268  million  (fifth  row)  per  satellite,  but  the  unit  cost  is  now  estimated  to  be 
$239  million  (fourth  row).  The  difference  between  the  two  is  accounted  for  by  the  fact  that  other  government 
costs  ended  up  $29  million  per  satellite  lower  than  estimated. 
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rose  58  percent  between  Block  I  and  Block  II  (from  $239  million  to  $377  million). 
Unit  costs  between  Block  II  and  Block  II  follow-on  (Ilf)  are  projected  to  rise  52  percent 
(from  $377  million  to  $574  million).  Table  6.2  indicates  when  each  WGS  satellite  was 
ordered,  when  each  was  delivered,  and  the  difference  in  years;  Figure  6.1  indicates  the 
interval  during  which  the  USAF-purchased  WGS  satellites  were  built  and  launched. 
The  table  indicates  a  large  gap  between  WGS  Block  I  and  WGS  Block  II  and  a  smaller 
gap  between  WGS  Block  II  and  WGS  Block  Ilf.  However,  the  time  between  program 
approval  and  launch  for  WGS  Block  I  was  five  to  seven  years,  and  the  expected  cycle¬ 
time  for  WGS  Block  II  is  shorter,  four  to  five  years.  If  current  launch  dates  for  Block  Ilf 
prove  accurate,  then  the  gap  between  Block  I  and  Block  II  will  be  somewhat  smaller 
than  the  gap  between  Block  II  and  Block  Ilf. 


Sources  of  Nunn-McCurdy  Breach 

The  WGS  cost  breach  has  two  components:  the  increase  in  unit  costs  between  Block  I 
and  Block  II  satellites  and  the  increase  in  unit  costs  between  Block  II  and  Block  Ilf 
satellites.  The  first  difference  was  ascribed4  to  “what  proved  to  be  an  artificially  low  cost 
for  the  original  three  vehicles  under  a  firm  fixed-price  contract.”  This  section  focuses 


Table  6.2 

WGS  Order  and  Launch  Years 


Satellite 

Budget 

Year 

Launch 

Year 

Difference 
in  Years 

1 

2002 

2007 

5 

Block  1 

2 

2002 

2009 

7 

3 

2003 

2009 

6 

4 

2007 

2011a 

5a 

Block  II 

5 

2008 

2012a 

4a 

6  (Aus.) 

2009 

2013 

4a 

Block  Ilf 

7 

2011 

2016 

5 

8 

2012 

2017 

5 

a  These  are  the  launch  dates  taken  from  the  2012  President's  Budget 
and  are  one  to  two  years  later  than  expected  launch  dates  were  at 
the  time  of  our  study  in  April  2010. 


4  Letter  from  the  Secretary  of  the  Air  Force  to  the  Chairman,  Senate  Committee  on  Armed  Services,  March  8, 
2010. 
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Figure  6.1 

WGS  Production/Launch  Periods 


2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016 

Year 

NOTES:  Launches  do  not  include  MexSAT-1  and  MexSAT-2.  Launch  dates  will  follow  MexSAT-3. 

RAND  MG1 171z1-6.1 

on  the  latter  cost  increase,  largely  because  it  is  the  current  one  and  thus  far  more  rel¬ 
evant  to  decisions  to  be  made  on  the  WGS  program. 


Examination  of  Unit  Cost  Increase  Between  Block  II  and  Block  Ilf 

Table  6.3,  whose  numbers  were  generated  by  the  Program  Office  (note  that  these  are 
Program  Office  estimates,  not  Boeing’s  actual  cost  proposals)  in  then-year  dollars, 
shows  both  blocks  in  terms  of  target  and  ceiling  costs.  The  latter  includes  margin  suf¬ 
ficient  to  account  for  the  possibility  of  cost  overruns  on  the  FPIF  work  (combining 
advanced  procurement,  base  procurement,  and  launch  support  costs). 


Table  6.3 

Program  Office  Unit  Cost  Breakdown  (Current 
Year,  $  Millions) 


BY 

Target 

Ceiling 

Block  II 

2007 

355 

410 

Block  Ilf 

2011 

— 

555 
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How  do  $555  million  and  $410  million  (in  current  dollars)  compare  with  the 
aforementioned  $574  million  and  $377  million  (in  BY  2001  $)?  Table  6.4  illustrates 
the  difference. 

Several  features  merit  note.  First  is  that  storage  and  factory  restart  costs  were 
very  small  in  going  from  Block  I  to  Block  II  but  substantial  in  going  from  Block  II  to 
Block  Ilf,  even  though  the  gap  before  restarting  production  was  four  years  for  Block  II 
and  only  two-and-a-half  years  for  Block  Ilf.  We  could  not  explain  this  difference  in  the 
time  allotted  for  the  study.  Second,  in  both  cases,  other  government  costs  (estimated 
using  data  from  the  Program  Office  and  SAF)  are  fairly  large  but  roughly  the  same. 
These  costs  include  contracting  office  and  engineering  costs;  it  was  estimated  by  sub¬ 
tracting  known  cost  components  from  total  cost  components  and  checked  for  overall 
reasonableness  and  consistency. 

Third,  and  most  important,  the  bottom  line  unit  price  figure  for  the  Block  II  sat¬ 
ellite  is  $355  million  rather  than  the  $410  million  ceiling  price.  The  reason  for  this  is 
that  the  $355  million  represented  the  contracted,  hence  targeted,  price  of  the  satellites; 
if  Boeing  costs  were  higher  than  $355  million,  then,  under  the  terms  of  the  contract, 
the  federal  government  would  reimburse  Boeing  only  for  80  percent  of  those  additional 
costs.  The  $410  million  was  the  ceiling  price;  Boeing  would  have  to  absorb  all  costs  in 
excess  of  that  amount.  Building  the  current  APB  APUC  (for  Blocks  I  and  II)  out  of  the 
contract  price  but  building  the  expected  APB  APUC  (for  Blocks  I,  II,  and  Ilf)  out  of 
the  ceiling  price  essentially  compares  apples  and  oranges.  In  effect,  the  WGS  Program 
Office  built  a  15  percent  hedge  factor  into  the  price.  We  cannot  explain  the  program¬ 
mers’  motivation  for  doing  so,  particularly  because  it  led  to  a  critical  Nunn-McCurdy 
breach  that  otherwise  could  have  been  avoided.  Whether  or  not  this  difference  repre¬ 
sents  their  lack  of  confidence  in  the  estimate  can  only  be  a  matter  of  speculation.  Were 
this  15  percent  removed,  then  the  unit  cost  of  Block  Ilf  would  have  been  $516  million 
(in  current  dollars)  rather  than  $574  million,  yielding  an  APUC  of  $357  million,  or  an 


Table  6.4 

Relating  Base  Year  and  Current  Year  Costs  ($  Millions) 


Block  II 

Block  Ilf 

Unit  cost,  BY  2001  $ 

377 

574 

Inflation  factor  to  current  costs 

1.14  (BY  2007) 

1.207  (BY  2011) 

Unit  cost  current  year  dollars 

430 

693 

Less  storage  and  factory  restart 

4 

73 

Subtotal 

426 

620 

Less  other  government  costs 

71 

65 

Subtotal  (from  Table  6.3) 

355 

555 
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increase  of  22  percent  rather  than  27  percent  (that  is,  a  “significant”  rather  than  “criti¬ 
cal”  breach). 

Nevertheless,  $555  million  is  still  a  substantial  increase  over  $355  million — and 
needs  to  be  explained.  Table  6.5  lists  the  various  factors. 

We  start  with  the  unit  price  of  $355  million.  Next  we  add  the  current  cost  over¬ 
run  of  3  percent  ($11  million)  (although  the  final  cost  overrun  may  be  higher  or  lower, 
we  presume  that  cost  overruns  experienced  to  date  establish  a  new  baseline  for  what  it 
really  costs  to  build  a  WGS):  hence,  $366  million.  The  next  adjustment,  line  4,  factors 
in  four  years  of  inflation  at  3.5  percent  per  year  (as  calculated  by  the  Program  Office 
based  on  historical  experience  in  satellite  component  and  manufacturing  costs);5  hence, 
the  $420  million  in  line  5.  Next  comes  $2  million  for  additional  tests  not  required  for 
Block  II,  $35  million  (as  calculated  by  Boeing)  to  pay  for  three  critical  components  that 
might  otherwise  go  out  of  production,6  and  $25  million  (also  as  calculated  by  Boeing) 
for  cost  increases  in  other  components  at  risk  in  the  supply  chain;  hence,  the  subtotal 
of  $482  million  in  line  9.  The  last  adjustment  arises  from  the  accounting  artifact  noted 
above — the  difference  between  contract  costs  used  to  calculate  Block  II  prices  and  the 
ceiling  cost  used  to  calculate  Block  Ilf  prices.  This  brings  us  to  the  $555  million  that 
the  Program  Office  uses  to  calculate  unit  costs  for  Block  Ilf. 


Table  6.5 

Cost  Increase  Between  Block  II  and  Ilf  (Current  Year,  $  Millions) 


Increase  Component 

Block  II 

1. 

Unit  cost,  BY  2007  $ 

355 

2. 

3%  cost  overrun 

11 

3. 

Actual  unit  cost,  BY  2007  $ 

366 

4. 

Four  years  of  inflation  at  3.5  percent  per  year 

1.147a 

5. 

Expected  unit  cost  circa  2011 

420 

6. 

Extra  tests 

2 

7. 

Higher  component  prices  for  three  items 

35 

8. 

Higher  component  prices  overall 

25 

9. 

Subtotal 

482 

10. 

Risk  premium  of  15  percent 

555 

a  1.035  x  1.035  x  1.305  x  1.035  =  1.147  x  $366  million  =  $420  million. 


5  Note  that  this  3.5  percent  exceeds  the  1.8  percent  used  as  an  overall  price  deflator  by  OSD  to  convert  constant 
into  current  dollars. 

6  The  xenon  ion  propulsion  system,  certain  transponders,  and  a  crypto  box. 
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Explaining  the  Cost  Differences 

The  $60  million  in  component  cost  inflation  (over  and  above  the  normal  3.5  percent 
a  year)  shown  in  rows  7  and  8  of  Table  6.5  requires  further  explanation.  Reflecting  a 
general  shift  in  market  requirements,  Boeing  shifted  its  commercial  satellite  offerings 
from  its  HS702HP  (high-power)  bus  to  its  HS702MP  (medium-power)  bus.  This  shift 
has  left  WGS  supporting  the  production  of  parts  that  no  longer  have  much  commercial 
demand,  thereby  raising  the  cost  of  these  components.  That  noted,  Boeing  also  claims 
that  the  cost  ratio  between  bus  and  payload  is  expected  to  remain  constant  and  that 
the  cost  ratio  between  component  costs  and  Boeing’s  costs  is  also  expected  to  remain 
constant.  Both  imply  that  its  internal  costs  have  also  risen  more  or  less  proportionately 
with  component  costs.  This  may  be  reflected  in  the  charges  associated  with  the  cold 
factory  restart  noted  above.  Figure  6.2  indicates  a  sharp  decline  in  commercial  satellite 
production  at  about  the  same  time  that  WGS  production  started.  Although  the  pace  of 


Figure  6.2 

Launch  Dates  for  Boeing-Produced  Satellites 
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satellite  construction  has  recovered,  it  has  not  returned  to  earlier  levels  that  character¬ 
ized  the  first  few  years  of  this  century. 

Component  cost  inflation  also  reflects  a  broader  phenomenon — the  growing 
divergence  between  WGS  and  its  civilian  counterpart.  Commercial  products  change 
constantly;  military  products  change  infrequently  (but  in  relatively  large  chunks)  and, 
in  the  case  of  military  specification  products,  may  not  change  at  all  precisely  because 
product  qualification  is  both  torturous  and  tortuous.  In  effect,  the  WGS,  born  as  a 
modification  to  a  commercial  business  line,  has  evolved  to  a  program  that  is  primarily 
military.  As  noted,  the  WGS  satellite  bus  has  diverged  from  its  civilian  counterpart. 
The  payload  of  the  WGS  satellite  consists  of  Ka-band  transponders  and  X-band  tran¬ 
sponders  and  channelizers  to  switch  between  the  two.  X-band  is  primarily  military  to 
begin  with.  The  commercial  market  flirted  with  Ka-band  ten  years  ago,  but  the  trend 
toward  terrestrial  (fiber  optics  and  cell  phones)  rather  than  satellite-based  communica¬ 
tions  has  put  a  damper  on  the  industry’s  interest  in  exploring  different  spectra  whose 
primary  virtue  is  that  it  is  largely  unclaimed.  Furthermore,  the  global  business  of  U.S. 
satellite  manufacturers  has  been  dampened  by  increasingly  stringent  application  of 
International  Traffic  in  Arms  Regulations  rules  starting  ten  years  ago.  Components 
that  once  could  be  supported  from  both  WGS  and  commercial  sales  increasingly  rely 
on  the  WGS  market,  and  suppliers  must  be  paid  a  premium  to  remain  in  the  market. 
Similarly,  former  WGS  workers  who  could  count  on  transferring  their  skills  into  very 
similar  commercial  work  when  gaps  appear  in  WGS  face  a  harder  transition.  The  days 
are  gone  when  commercial  sales  could  buoy  the  resources  put  into  the  WGS  program 
between  one  buy  and  the  next.  The  economics  of  WGS  increasingly  depend  on  the 
pace  and  scheduling  of  WGS  buys  alone. 


Boeing  and  the  Commercial  Satellite  Market 

This  section  examines  some  of  the  background  trends  in  the  commercial  satellite  busi¬ 
ness  together  with  Boeing’s  position  within  the  market. 

Aviation  Week  &  Space  Technology  estimated  that  military  satellite  orders  for  both 
Lockheed  Martin  and  Boeing  over  the  next  ten  years  would  be  for  29  satellites  for 
$11.7  billion  for  Lockheed  Martin  and  25  spacecraft  for  over  $5  billion  for  Boeing.7 
This  suggests  that  manufacturing  military  satellites  will  cost  between  $200  million 
and  $400  million  each.  This  is  well  below  the  WGS  Block  Ilf  unit  costs,  although  sat¬ 
ellites,  it  should  be  cautioned,  differ  widely  in  their  individual  capabilities. 

Ka-band  satellite  transmission  growth  rate  has  slowed  over  the  past  few  years,  as 
evident  in  Table  6.6,  declining  from  some  40  percent  to  about  2  percent. 


7  John  S.  Edwards,  “Military  Space  Becomes  Increasingly  Important  as  Capabilities  Mature,”  Aviation  Week  & 
Space  Technology ,  January  25,  2010,  p.  122. 
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Table  6.6 

Ka-Band  Transponder  Recent  Trends 


Fiscal  Year 

Number  of 
Transponder 
Equivalents3 
Launched 

Number  of 
Ka-Band 
Transponder 
Equivalents 
Launched 

Percentage  of 
Ka-Band 
Transponder 
Equivalents 
Launched 

2007 

-730 

-300 

41 

2008 

-750 

-100 

13 

2009 

-850 

-20 

2 

SOURCE:  "Futron  Satellite  Telecommunications  2009  Year-End  Summary,"  Aviation 
Week  &  Space  Technology,  undated. 
a  A  proxy  measure  for  communications  throughput. 


However,  “Both  Hughes  [Boeing]  and  ViaSat  have  ordered  large  new  Ka-band 
satellites  to  keep  up  with  demand.  ViaSat-1  is  scheduled  for  launch  in  February  2011, 
with  Hughes’  [Boeing’s]  Jupiter  satellite  scheduled  for  launch  in  2012.”8  Space  Systems/ 
Loral  is  constructing  both  systems.  ViaSat-1  is  estimated  to  cost  $300  million  for  con¬ 
struction,  launch,  and  insurance.9 

XTAR  is  providing  commercial  X-band  service  to  Spain,  Denmark,  Germany, 
the  U.S.  State  Department,  and  Defense  Information  Systems  Agency. 

In  a  2003  review  of  geosynchronous  commercial  satellite  buses,  Futron  found 
“Boeing’s  BS601  and  BS702  performance  complexity  and  problems  along  with  declin¬ 
ing  levels  of  customer  support  cited  consistently  as  causing  their  low  ratings.”10  Addi¬ 
tionally,  “DirectTV  in  2004  ordered  three  Boeing  702  model  satellites  for  25  percent 
of  the  year’s  market  (along  with  options  for  a  fourth  satellite).  The  timing  probably 
kept  Boeing  Satellite  Systems  [BSS]  in  business.  Its  struggles  stemmed  from  on-orbit 
problems  with  its  601  and  702  series  satellites,  but  since  those  problems  were  identified, 
Boeing’s  hard  work  to  restore  confidence  in  its  manufacturing  unit  paid  off.”* 11 

Ultimately,  however,  rather  than  the  commercial  market  supporting  Boeing’s  702 
bus,  it  appears  that  government  business  has  been  supporting  the  system. 

In  its  10-year  history,  the  Boeing  division’s  main  platform,  the  702,  has  commonly 
served  big  commercial  requirements,  such  as  the  three  current  orders  for  DirecTV 
and  two  for  Sky  Terra.  But  the  platform  also  has  been  used  for  many  of  the  compa- 


8  Peter  de  Selding,  “ViaSat  Bullish  on  Ka-band,  Might  Order  2nd  Satellite  This  Year,”  Space  News,  February  9, 
2010. 

9  Typical  launch  and  insurance  costs  of  a  communication  satellite  are  roughly  $100  million;  WGS  costs  listed 
do  not  include  launch  costs. 

10  Futron,  GEO  Commercial  Satellite  Bus  Operations:  A  Comparative  Analysis,  August  13,  2003. 

11  John  S.  Edwards,  “Waiting  for  Rebound,”  Aviation  Week  &  Space  Technology,  January  16,  2006,  p.  161. 
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ny’s  major  government  programs,  most  prominently  the  Wideband  Global  Satcom 
(WGS)  network  of  six  spacecraft  that  replaces  the  Defense  Satellite  Communica¬ 
tions  System.  .  .  .  WGS  and  two  other  major  government  programs — the  Global 
Positioning  System  IIF  and  GOES  N-P  series — have  provided  90  percent  of  Boe¬ 
ing’s  recent  work.  To  redress  that  imbalance,  the  company  began  looking  for  new 
commercial  market  entries  four  years  ago  and  concluded  it  could  take  advantage 
of  the  702’s  flight  software,  avionics  and  power  management  systems  to  develop  a 
smaller  bus.12 


Root  Cause  Analysis 

Table  6.7  provides  the  WGS  version  of  the  PARCA  root  cause  narrative  matrix  (“X” 
indicates  that  the  category  is  relevant  for  that  year  but  there  is  nothing  to  report  on; 
the  text  gives  more  detail  in  place  of  an  “X”;  blanks  indicate  that  the  category  is  not 
relevant  for  that  year). 

The  52  percent  increase  between  Block  II  and  Block  Ilf  unit  pricing  is  primarily 
due  to  the  first  three  factors  listed  in  Table  6.8.  Such  results  are  necessarily  limited  by 
the  60-day  window  allowed  for  investigation  under  the  Nunn-McCurdy  legislation 
that  curtailed  RAND’s  ability  to  question  subcontractors  and  analyze  many  of  the  cost 
claims  that  had  to  be  accepted  as  valid  over  the  course  of  the  analysis. 

The  largest  factor — almost  one-third  of  the  increase — is  an  accounting  artifact 
where  the  Block  Ilf  prices,  as  calculated  by  the  Program  Office  and  as  thereby  dis¬ 
played  for  the  purposes  of  making  a  Nunn-McCurdy  presentation,  include  a  15  percent 
risk  premium,  whereas  Block  II  unit  costs  do  not  (because  they  largely  reflect  expended 
rather  than  projected  costs).  This  results  in  an  apples  and  oranges  comparison.  Inas¬ 
much  as  the  Block  Ilf  is  practically  identical  to  the  Block  II  units  that  Boeing  is  already 
building,  Boeing  can  be  realistically  expected  to  produce  the  satellites  at  near  the  target 
cost,  which  is  15  percent  below  the  ceiling  cost — although  Block  II  is  running  3  per¬ 
cent  over  target.  But  the  ceiling  price  is  what  was  reported.  Next,  Boeing  is  charging 
for  storage  and  restart  costs  for  the  2.5  year  hiatus  between  Blocks  II  and  Ilf.  On  the 
surface,  the  cause  appears  to  be  the  interruption  in  production,  but  the  four-year  hiatus 
(measured,  as  noted,  in  terms  of  when  satellites  were  ordered,  not  when  they  launched) 
between  Block  I  and  Block  II  had  a  cost  of  only  $3.5  million,  or  less  than  7  percent  of 
the  current  estimate.  One  explanation  is  that  significant  aspects  of  WGS  production 
are  no  longer  supported  by  the  commercial  market  and  therefore  require  storage  and 
restart  expenses  during  production  breaks.  Finally,  key  components  of  WGS  that  are 
no  longer  supplied  to  the  commercial  market  will  have  greatly  increased  procurement 
costs  accounting  for  another  26  percent  of  the  cost  increase.  The  second  and  third  fac- 


12 


Michael  Mecham,  “Hosted  Payloads  Growth  Eyed  by  U.S.  Satellite  Makers,”  December  7,  2009,  p.  66. 


Table  6.7 

PARCA  Root  Cause  Narrative  Matrix  for  the  WGS 


Year  from  MS  B  and  Fiscal  Year 

B 

2001 

1 

2002 

2 

2003 

3 

2004 

4 

2005 

5 

2006 

6 

2007 

7 

2008 

8 

2009 

9 

2010 

10 

2011 

Baseline  issues 

Unrealistic  estimates 
for  cost  or  schedule 

BSS  claims 
losing  $  on 
Block  1  (WGS 
1-3) 

X 

X 

X 

X 

BSS  re¬ 
prices 
Block  II 
(WGS  4,  5) 

X 

X 

X 

X 

BSS 

expected 
to  re-price 
Block  Ilf 
(WGS  7,  8) 

Immature 

technology;  excessive 
manufacturing, 
integration  risk 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Unrealistic 

performance 

expectations 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Execution  issues 

Changes  in 

procurement  quantity 

X 

WGS  4,  5 
added 

WGS  7,  8 
added 

Inadequate  funding/ 
funding  instability 

X 

Unanticipated 
design,  engineering, 
manufacturing,  or 
technical  issues 

X 

Poor  performance 
of  government  or 
contract  personnel 

X 

Other 
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Table  6.8 

Primary  Factors  for  Block  II  to  Block  Ilf  Unit  Cost  Increase  (BY  2001  $  Millions) 


Factor 

Increase 

Amount 

Percentage 

Risk  premium  accounting  artifact 

60 

30 

Storage  and  restart  costs 

57 

29 

Increased  component  costs 

51 

26 

Other  (e.g.,  SATCOM  industry  inflation,  cost  overruns) 

29 

15 

tors  support  the  argument  that  the  root  causes  of  the  breach  are  changes  in  the  com¬ 
mercial  market  without  corresponding  changes  in  the  WGS  design  and  procurement. 

Despite  these  large  cost  increases,  the  WGS  program  is  essentially  healthy  and 
relatively  well  managed.  The  satellites  work;  three  of  them  are  already  on  orbit  serving 
customers.  These  customers  are  generally  happy,  which  is  part  of  the  reason  that  the 
currently  planned  WGS  constellation  is  larger  than  the  one  originally  planned  (more 
often,  total  buys  decline  over  the  life  of  a  contract).  There  is  no  reason  to  expect  that 
the  cost  of  subsequent  satellites  after  WGS  8  will  increase;  quite  the  contrary.  Boeing’s 
bid  proposals  for  WGS  9  through  12  suggest  that  they  will  run  $100  million  less  than 
WGS  7  did  (once  due  account  is  taken  of  the  baseline  inflation  in  the  satellite  indus¬ 
try).  Thus,  although  the  cost  increases  in  what  should  be  a  stable  program  may  appear 
startling  (and  remain  somewhat  startling  even  after  explanation),  this  is  no  indicator 
of  a  program  facing  technological  or  production  problems  that  cannot  be  reasonably 
solved. 

The  broader  lesson  learned  for  this  program  is  that  when  DoD  procurement  pig¬ 
gybacks  on  a  commercial  base,  notably  the  commercial  base  of  a  particular  company 
and  its  ecosystem,  it  takes  a  certain  risk.  The  base  may  shrink,  leaving  it  with  less 
capacity  to  cover  total  overhead  costs.  Even  if  the  base  does  not  shrink,  it  will  evolve.  If 
DoD  requirements  do  not  evolve  in  parallel — and  there  is  no  inherent  reason  why  they 
should — the  divergence  between  DoD’s  requirements  and  the  market’s  requirements 
means  that  either  the  requirements  are  compromised  (admittedly,  this  may  be  accept¬ 
able  in  some  circumstances)  or,  eventually,  such  programs  have  to  stand  on  their  own 
feet.  They  can  no  longer  be  free  riders,  so  to  speak.  This  suggests  that  a  certain  procure¬ 
ment  discipline  is  called  for,  or  DoD  will  pay  the  difference.  Start-stop  programs  are 
costlier  than  steady-state  programs  (i.e.,  when  buys  are  consistent  from  one  year  to  the 
next),  which,  in  turn,  are  somewhat  more  costly  than  total  buy  programs  (e.g.,  we  want 
six  satellites,  deliver  them  when  you  finish  them).  Although  DoD  cannot  necessarily 
commit  to  even  procurements  for  a  variety  of  reasons  (e.g.,  changing  requirements,  risk 
management,  congressional  politics),  everyone  concerned  should  understand  that  there 
are  costs  entailed  in  maximizing  acquisition  flexibility. 
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Conclusions 

Three  primary  factors  contribute  to  the  Nunn-McCurdy  breach:  an  accounting  arti¬ 
fact,  increase  in  the  cost  of  component  parts,  and  storage  and  restart  costs.  Each  con¬ 
tributes  to  about  one-third  of  the  cost  increase  between  Block  II  and  Ilf.  An  underlying 
factor  of  the  increase,  particularly  with  respect  to  the  storage  and  restart  costs,  is  the 
change  that  occurs  in  the  commercial  product  base  that  affected  the  WGS  costs.  The 
government  incurred  additional  costs  because  the  commercial  base  of  Boeing  no  longer 
supported  the  WGS.  This  probably  would  not  have  occurred  if  all  Boeing  had  to  do 
was  pull  parts  from  an  active  commercial  line.  Thus,  when  the  government  links  one  of 
its  programs  to  a  company’s  commercial  base,  it  assumes  an  additional  measure  of  risk. 


CHAPTER  SEVEN 

Conclusions 


This  report  addresses  program  specifics,  but  some  common  observations  can  be  made 
about  the  analysis  performed.  At  a  minimum,  three  fundamental  elements  are  required 
to  perform  the  analysis  in  a  fashion  that  meets  the  congressional  demands  and  expec¬ 
tations  discussed  in  Chapter  Two.  These  are  (1)  access  to  adequate  sources  of  raw  data 
on  programs,  (2)  access  to  individuals  in  both  government  and  industry  to  gain  the 
fullest  possible  understanding  of  both  data  and  program  execution,  and  (3)  adequate 
numbers  of  informed  and  experienced  research  staff  to  allow  the  effort  to  be  completed 
in  time  and  consistent  with  the  needs  of  the  Secretary  of  Defense  to  comply  with  the 
law.  These  three  elements  also  form  the  basis  of  our  recommendations. 

As  was  apparent  in  the  program-specific  chapters,  each  of  the  four  programs 
investigated  by  RAND  had  different  execution  specifics,  reflecting  diverse  com¬ 
plexities.  However,  at  summary  levels,  RAND’s  root  cause  analysis  identified  several 
common  causes  of  threshold  breaches.  Table  7.1  displays  how  each  program  analyzed 
reflected  the  three  basic  summary  causes  for  breach:  planning,  changes  in  economy, 
and  program  planning.  The  comparison  matrix  lists  several  subcategories;  the  list  is 
not  exhaustive  but  it  contains  the  best  that  could  be  constructed  as  general  categories, 
given  the  level  of  detail  in  the  data  available. 


Access  to  Data 

The  data  that  underpinned  the  analysis  that  supports  these  comparisons  came  from  the 
breach  notifications,  SARs,  acquisition  strategy,  and  decision  documents  and  briefings 
that  relate  to  each  program.  These  source  materials  were  discussed  in  greater  detail  in 
Chapter  One.  Although  they  were  sufficient  to  complete  this  first  root  cause  analysis, 
going  forward,  more  thought  and  effort  will  be  required  to  ensure  that  appropriate 
detail  and  relevant  material  are  available  at  the  start  of  the  analytic  effort,  given  the 
short  time  allowed  by  the  statutorily  imposed  deadlines.  In  that  context,  the  approach 
developed  by  the  government  to  perform  the  root  cause  analysis  has  great  significance 
for  the  necessary  advance  planning  and  determinations  necessary  to  best  extract  the 
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Table  7.1 

Comparison  Matrix  of  Root  Causes  of  Program  Cost  Growth 


Category 

Root  Cause  of  Nunn-McCurdy  Breach 

WGS 

Apache 

DDG-1000 

JSF 

Planning 

Underestimate  of  baseline  cost 

V 

V 

V 

V 

Ambitious  scheduling  estimates 

V 

V 

Poorly  constructed  contractual  incentives 

H 

V 

Immature  technologies 

vv 

V 

vv 

Ill-conceived  manufacturing  process 

V 

Unrealistic  performance  expectations 

V 

Delay  in  awarding  contract 

V 

Insufficient  RDT&E 

V 

V 

V 

V 

Changes  in 

Increase  in  component  costs 

H 

V 

V 

V 

economy 

Increase  in  labor  costs 

V 

V 

Discontinued/decreased  production  of  components 

V 

Decreased  demand  for  similar  technology  in  private 
sector  (economies  of  scale) 

V 

Inflation 

V 

V 

V 

V 

Production  delays 

H 

V 

vv 

Change  in  procurement  quantities 

Increase 

V 

vv 

Decrease 

vv 

V 

Program 

Unanticipated  design,  manufacturing,  and 

V 

vv 

management 

technology  integration  issues 

vv 

Lack  of  government  oversight  or  poor  performance 
by  contractor  personnel 

V 

V 

Inadequate  or  unstable  program  funding 

V 

V 

V 

V 

Accounting  artifact 

V 

NOTE:  V  =  Root  cause,  V  V  =  Significant  root  cause. 


nature  of  the  research  capability  value  from  the  supporting  FFRDC  or  academic  par¬ 
ticipants  in  the  analysis  and  certification  process. 

The  level  of  effort  reflected  in  Chapters  3-6  and  the  similar  effort  at  other 
FFRDCs  and  the  University  of  Tennessee  were  deemed  by  the  PARCA  director  from 
the  outset  to  exceed  what  the  government  could  do  in  the  time  available.  Because  of 
the  inherently  governmental  nature  of  the  effort,  the  use  of  FFRDCs  in  particular  was 
an  effective  strategic  approach  for  the  PARCA  director  and  appears  to  be  one  that  will 
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continue  to  be  used.  Going  forward,  participants’  efforts  in  performing  these  first  pro¬ 
gram  assessments  required  by  the  statute  should  be  used  to  strengthen  the  demand  to 
make  more  data  available.  For  example,  some  internal  decision  material  not  normally 
releasable  would  be  useful  in  performing  analysis.  Given  the  nature  of  the  relationship 
between  the  government  and  FFRDCs,  these  data  should  be  released,  because  such 
release  will  both  speed  the  analysis  and  improve  its  quality.  A  notional  listing  of  desir¬ 
able  data  is  shown  in  Table  7.2. 

The  following  list  includes  document  types  that  RAND  and  other  FFRDCs 
would  request  from  the  PARCA  office  at  the  beginning  of  each  study  and  the  reason 
for  each  request.  Each  item  would  be  used  in  the  analysis  and  would  provide  insight 
into  the  program  from  different  points  of  view. 

We  will  need  additional  documents  as  the  project  progresses: 

•  program  and  service  briefings  from  the  preceding  year  to  give  an  idea  of  the  state 
of  the  program  over  the  past  year  or  so 

•  OIPT  briefings  from  all  five  teams 

•  program  manager  briefings  to  the  IPTs. 

As  long  as  the  FFRDCs  have  access  to  the  DAMIR  system,  they  will  download  their 
own  copies  of  the  following: 

SARs 

•  official  description  of  the  information  history  of  the  program,  as  reported  to 
Congress 

DAES 

•  current  “official”  program  status. 

Table  7.2 

Data  Source  Needs 


Document  Type 


Reason 


Notifications  to  Congress  This  will  allow  the  FFRDC  to  study  exactly  why  the  Nunn- 

McCurdy  Act  was  breached,  as  it  was  reported  to  Congress 

Latest  signed  acquisition  strategy  Foundation  of  what  the  current  strategy  is  so  that  the  team 

has  an  up-to-date  starting  point 


DAB  and  IPR  materials 


Support  materials  from  these  will  assist  in  showing  the 
decisions  that  affect  the  program  throughout  its  life 


ADMs 

Program  deviation  reports 
Nunn-McCurdy  OIPT  kick-off  briefing 


Description  of  the  official  parameters  that  govern  a  program 

Record  of  spending  deviations  from  the  original  program  plan 

Strategic  guidance  and  issue  resolution  for  cost  issues  and 
management  issues 


All  previously  signed  acquisition  This  will  allow  the  team  to  see  how  the  program  has  evolved 

documents  overtime 
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This  additional  material,  in  conjunction  with  greater  access  to  program,  government, 
and  contractor  managers,  would  enable  a  fuller  examination  of  the  issues  identified  in 
Table  1.2,  which  are  more  expansive  than  those  represented  in  the  comparison  matrix 
in  Table  7.1. 

These  additional  data  would  also  give  greater  fidelity  to  the  identification  of  issues, 
for  both  the  RCA  and  performance  assessment  elements  of  the  statutory  requirements. 
As  RAND  examined  metrics-based  approaches  to  supporting  the  PARCA  director  in 
his  full  range  of  reporting  responsibilities,  we  found  the  current  approaches  being  con¬ 
sidered  to  be  incomplete.  The  gap  analysis  we  performed  and  shared  with  the  PARCA 
director  and  his  staff  can  be  found  in  Appendix  B.  We  believe  that  subsequent  analysis 
efforts  will  identify  further  gaps.  As  DoD  appears  committed  to  using  a  standardized 
metrics  approach  to  enable  better  understanding  of  program  execution,  we  believe 
that  it  is  necessary  to  develop  a  comprehensive  set  of  metrics  to  give  transparency  to 
program  execution,  avoid  subjective  judgment,  and  avoid  the  wasting  of  time  in  both 
executing  commands  and  in  oversight  offices.  This  is  consistent  with  the  fundamental 
recommendations  of  the  Packard  Commission  and  Secretary  Robert  Gates’s  initiative 
to  eliminate  inefficiency  and  waste. 

Examples  of  the  kind  of  data  gaps  that  have  become  apparent  are  those  company 
financial  status/strength  and  business  base  (commercial  and  international)  issues  that 
we  saw  in  play  on  both  the  satellite  and  aircraft/ship  platform  programs.  In  the  sum¬ 
mary  of  this  report,  we  refer  to  Loren  Thompson’s  observation  on  Army  trucks  being 
produced  at  Oshkosh.  Looking  to  the  smaller  ship  builders  such  as  Marinette  Marine 
and  Austal  on  the  littoral  combat  ship,  one  has  to  consider  the  effect  of  quantity  reduc¬ 
tions,  such  as  those  of  the  DDG-1000,  on  their  financial  viability.  Line-of-businesses 
considerations  by  the  satellite  manufacturers  played  a  large  role  in  the  WGS  root  cause 
analysis  and  was  driven  by  commercial  business  decisions.  The  challenges  reported 
on  the  Advanced  Extremely  High  frequency  1  launch  and  stationing  on  August  14, 
2010, 1  reveal  the  uncertain  nature  of  a  business.  Although  Lockheed  clearly  has  a  more 
robust  business  base,  even  it  can  be  affected  in  a  manner  that  leads  to  unavoidable  cost 
increases  and  schedule  delays  in  government  programs.  Similar  to  commercial  market 
implications  is  the  significance  of  international  sales  on  a  contractor’s  ability  to  main¬ 
tain  cost  and  schedule  targets.  Lor  example,  in  our  review  of  the  JSL,  we  noted  that 
although  international  sales  had  not  been  entered  into  the  threshold  calculations,  these 
sales  represented  a  potentially  large  segment  of  Lockheed’s  business  base  going  forward 
and  could  produce  volatility.  Since  our  quick  look  in  early  summer  2010,  both  the 
Dutch  and  British  governments  have  experienced  difficulty  with  maintaining  levels  of 
defense  spending.  If  sales  to  these  countries  do  not  materialize  as  scheduled,  there  is 
the  potential  of  a  threshold  breach  in  the  JSL  program.  When  coupled  with  the  poten¬ 
tial  technical  and  production  slippages  addressed  by  Lockheed  Martin  CEO  Robert 
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‘Military  Satellite  Relying  on  Backup  to  Save  Itself,”  Spaceflight  Now,  August  30,  2010. 
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Stevens  in  September  2010,  the  implication  for  program  cost  and  schedule  could  be 
significant.2 

The  point  of  all  this  is  to  say  that  some  higher-order  tools  may  need  to  be  used 
and  metrics  developed  to  gain  better  insights.  And,  in  the  process,  an  evaluation  of 
contractor  efficiency  in  applying  labor  and  capital  to  deliver  a  product  may  provide 
important  insights  and  warnings.  To  assess  the  relative  performance  of  major  weapon 
systems  producers,  defense  officials  can  apply  data  envelopment  analysis — a  quantita¬ 
tive  technique  that  relies  on  linear  programming  that  has  been  applied  in  numerous 
manufacturing  and  service  industries  to  compare  the  relative  efficiency  of  firms  con¬ 
verting  inputs  (e.g.,  labor,  capital,  and  raw  materials)  into  outputs. 

Access  to  Government  and  Contractor  Personnel 

More  transparent  and  meaningful  discussions  with  government  and  contractor  offi¬ 
cials  are  an  important  second  step.  The  better  these  individuals  understand  the  queries 
and  specific  aspects  of  the  basis  of  questions,  the  more  useful  these  discussions  will 
be.  As  noted  in  Chapters  Three  through  Six,  all  members  of  the  RAND  team  found 
exchanges  of  views  and  understanding  to  be  critical  to  our  ability  to  fully  support  DoD 
in  performing  its  inherently  governmental  function.  Clearly,  government  and  contrac¬ 
tor  team  time  is  valuable.  The  IPT  process  used  in  the  first  round  of  these  RCAs  was 
an  effort  to  limit  repetitive  visits  and  questions  from  multiple  sources,  all  intended  to 
illuminate  the  same  issues.  If  that  activity  is  deemed  to  be  successful  and  is  repeated 
in  future  RCAs,  every  effort  should  be  made  to  regularize  IPT  activity  and  allow  full 
engagement  by  all  parties  as  appropriate  if  that  becomes  the  principal  path  for  gaining 
greater  understanding  of  project/program  activity.  Clearly,  a  lot  of  activity  had  to  be 
packed  into  a  few  weeks  to  meet  the  Nunn-McCurdy  time  lines.  The  more  that  system¬ 
atic  approaches  for  inquiry  can  be  developed,  the  better  the  opportunity  for  success. 


Adequate  Research  Staff 

The  RAND  team  found  that  being  able  to  assemble  and  mobilize  a  group  of  experi¬ 
enced  researchers  with  some  knowledge  of  the  technical  disciplines  involved  quickly 
paid  great  dividends.  Our  first  experiences  showed  that  maintaining  that  type  of  capa¬ 
bility  over  time  will  take  continuing  commitment  and  active  cooperation  between  the 
government  and  its  FFRDC  and  academic  partners.  Engaging  the  RAND  analyst 
team  in  the  continuing  performance  assessment  process  is  proving  to  be  of  great  benefit 
to  having  available  knowledgeable  cadres  that  can  respond  quickly  to  emerging  issues. 


2  “Lockheed  CEO:  F-35B  ‘Re-phasing’  Possible,”  Aviation  Week  &  Space  Technology,  September  13,  2010. 
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The  application  of  gaming  theory  to  gauge  manager  responses  and  value  propositions 
should  provide  more  insights  on  areas  that  need  to  be  better  understood. 


Findings  and  Recommendations 

Table  7.3  summarizes  the  root  causes  of  the  Nunn-McCurdy  breaches  for  the  four 
programs  in  the  second  column.  The  middle  column  refers  to  issues  that  the  research 
teams  thought  might  be  influential  but  lacked  the  time  to  research  thoroughly.  These 
issues  warrant  additional  research.  The  right  column  contains  the  lessons  learned  from 
each  case. 

Each  root  cause  analysis  differs  somewhat  from  others  as  indicated  in  this  table. 
Although  quantity  changes  appear  to  lead  the  pack,  the  table  reveals  multiple  causes 
and,  with  further  time  and  effort,  RAND  could  develop  a  more  complete  view  of 
root  causes.  Potentially,  after  we  perform  a  number  of  these  analyses,  some  common 
threads  will  appear  and  we  may  see  some  consistency.  However,  in  the  meantime, 
Table  7.3  summarizes  what  has  been  gleaned  to  date. 

Table  7.3 

Summary  of  Root  Cause  Analysis 


Root  Cause  of 

Program  Nunn-McCurdy  Breach 


Issues  Requiring 
Further  Investigation 


Lessons  Learned 


Wideband 

Global 

Satellite 


Unanticipated  increase  in 
procurement  quantity 
Unstable  program  funding 
Underestimation  of  baseline  cost 
Increase  in  component  costs 
Production  delays  leading  to: 

(a)  storage  and  restart  expenses 

(b)  cessation  of  commercial 
market  support  for  production 
aspects 

Change  in  private  sector  market 
conditions 

Poorly  constructed  contractual 
incentives  for  contractor 
(distorted  reporting  of  incurred 
costs) 

Accounting  artifact 


Role  of  commercial  business 
reorganization  and  accounting 
cost-shifting  in  fueling  significant 
increases  in  storage  and  restart 
expenses 

Estimation  of  baseline  cost  for 
Block  I  program  (not  required  to 
be  reported  to  DoD) 


Production  delays  increase 
exposure  to  changing  private 
sector  market  conditions 
Acquisition  flexibility  (e.g.,  start- 
stop  programs)  comes  with  a  cost 


Longbow  Increase  in  procurement  quantity 

Apache  Underestimation  of  baseline  cost 

(training  and  technological 
development) 

Immature  technologies 
Unanticipated  redesign  and 
technology  integration  costs 
Increase  in  component  costs 
Increase  in  labor  costs 


Cost  estimates  should  include 
"contingency  dollars"  for 
unanticipated  design  changes  and 
technological  integration  issues 
Cost  estimates  should  be 
conducted  independently  of 
program  manager 
Combining  remanufactured  and 
new  build  items  cause  complexity 
and  cost  growth 
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Table  7.3 — Continued 


Root  Cause  of 

Program  Nunn-McCurdy  Breach 


Issues  Requiring 
Further  Investigation 


Lessons  Learned 


DDG-1000 


Decrease  in  procurement  Small  government  staff  size  to  Greater  planning  of 

quantity  monitor  program  manufacturing  process 

Inadequate  program  funding  and  Funding  instability  (and  its  effect  organization  required 


funding  instability 
Underestimation  of  baseline  cost 
and  scheduling 
Immature  technologies 
Unanticipated  manufacturing 
process  complexity  and 
technology  integration  issues 
(lack  of  precedent  or  prior 
experience) 

Unrealistic  performance 
expectations 

Small  government  staff  size  to 
monitor  program 
Production  delays 
Delay  in  awarding  contract 


on  program  execution)  caused  by: 

(a)  Navy  pressuring 
contractors  to  lower  price 
to  fit  target  budget 

(b)  Navy  actions  (as  opposed 
to  OSD  or  Congress) 

Effect  of  integration  risks  on 
production  delays  and  cost 
growth 


Large  reductions  in  procurement 
quantities  can  significantly 
increase  per  unit  cost 


Joint  Strike 

Decrease  in  procurement 

Use  of  common  parts  and 

Sufficient  RDT&E  required  to 

Fighter 

quantity 

suppliers  to  reduce  cost 

ensure  produceability  of  program 

Underestimation  of  baseline  cost 

Engine  redesign  issues 

Greater  government  oversight 

and  scheduling 

required  in  a  technologically 

Production  delays 

complex  project 

Immature  technology 
Unanticipated  design  issues 
Unanticipated  manufacturing 
process  complexity  and 
technology  integration  issues 

Lack  of  government  oversight  and 
poor  performance  by  contractor 
personnel 

More  hedges  against  risky 
elements  of  program  required 

Additional  collaboration  needed 
on  design  specifications  and 
discussion  of  cost-performance 
trade-offs 

Increase  in  component  costs 

Increase  in  labor  costs 

Insufficient  RDT&E 

Poorly  constructed  contractual 
incentives  for  contractor 

The  initial  effort  on  the  first  handful  of  Nunn-McCurdy  breaches  was  a  great 
start  in  arriving  at  a  systematic,  efficient,  and  economical  approach  to  doing  the  neces¬ 
sary  research  and  analysis  to  support  the  Secretary  of  Defense  in  his  Nunn-McCurdy 
Act  reporting  and  certification  responsibilities.  We  recommend  the  following  addi¬ 
tional  steps: 

•  the  establishment  of  a  comprehensive  set  of  material  to  be  used  for  analysis  and 
the  identification  of  and  access  to  that  material  for  all  engaged  parties 

•  the  establishment  of  a  clear  understanding  of  the  type  of  dialogue  needed  between 
all  engaged  parties  doing  research  for  the  government’s  purposes  and  program 
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personnel  to  limit  waste  of  time  and  duplication  of  effort;  using  new  analytic  tools 
or  decisions  games  tailored  to  the  effort  at  hand  may  better  inform  that  effort 
•  use  of  research  and  analysis  personnel  to  engage  on  MDAP  potential  fault  lines 
across  both  RCA  and  PA  in  a  continuum  of  effort  to  establish  knowledgeable  and 
capable  cohorts  in  both  the  process  of  breach  reporting  and  certification  as  well 
as  the  technologies  involved  in  MDAP  programs  to  be  scrutinized  and  assessed. 

In  the  entire  process,  greater  understanding  must  be  reached  of  what  really  mat¬ 
ters  in  program  execution  and  what  risk  areas  most  deserve  attention  on  a  continuing 
basis. 


APPENDIX  A 


Legal  Definitions  Applicable  to  Unit  Cost  Growth  Breaches 


A  unit  cost  growth  breach  occurs  when  unit  cost  growth,  as  computed  by  formulas 
defined  in  U.S.  law,  reach  or  exceed  thresholds  also  specified  in  federal  law.  The  thresh¬ 
olds  are  expressed  in  terms  of  the  legal  definitions  of  “program  acquisition  unit  cost” 
and  “procurement  unit  cost.” 


Definition  of  Program  Acquisition  Unit  Cost 

10  USC  §  2432  defines  the  term  “program  acquisition  unit  cost”  with  respect  to  a 
major  defense  acquisition  program  as  the  amount  equal  to  (a)  the  total  cost  for  devel¬ 
opment  and  procurement  of,  and  system-specific  military  construction  for,  the  acquisi¬ 
tion  program,  divided  by  (b)  the  number  of  fully  configured  end  items  to  be  produced 
for  the  acquisition  program. 

Showing  the  interrelationships  of  the  various  tools  that  Congress  has  mandated 
in  its  efforts  to  effect  better  acquisition  management,  SARs  have  adopted  the  acronym 
“PAUC”  for  program  acquisition  unit  cost. 

Definition  of  Procurement  Unit  Cost 

10  USC  §  2432  defines  the  term  “procurement  unit  cost”  with  respect  to  a  major 
defense  acquisition  program  as  the  amount  equal  to  (a)  the  total  of  all  funds  pro¬ 
grammed  to  be  available  for  obligation  for  procurement  for  the  program,  divided  by 
(b)  the  number  of  fully  configured  end  items  to  be  procured. 

SARs  have  also  adopted  the  acronym  “APUC”  for  acquisition  procurement  unit 
cost.  By  performing  the  computation  used  in  the  formulas  specified  in  10  USC  §  2432, 
the  APUC  in  the  SARs  is  the  same  entity  as  the  procurement  unit  cost  defined  in  10 
USC  §  2432. 

Breach  Thresholds 

10  USC  §  2433  defines  two  thresholds  for  unit  cost  growth.  These  are  a  significant  unit 
cost  growth  threshold  and  a  critical  unit  cost  growth  threshold. 
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A  significant  cost  growth  threshold  breach  is  defined  as 

•  a  PAUC  that  is  least  15  percent  over  the  program  acquisition  unit  cost  shown  in 
the  current  baseline  estimate  for  the  program  or 

•  an  APUC  that  is  least  15  percent  over  the  acquisition  procurement  unit  cost 
shown  in  the  current  baseline  estimate  for  the  program  or 

•  a  PAUC  that  is  at  least  30  percent  over  the  program  acquisition  unit  cost  shown 
in  the  original  baseline  estimate  or 

•  an  APUC  that  is  at  least  30  percent  over  the  acquisition  procurement  unit  cost 
shown  in  the  original  baseline  estimate. 

A  critical  cost  growth  threshold  breach  is  defined  as 

•  a  PAUC  that  is  least  25  percent  over  the  program  acquisition  unit  cost  shown  in 
the  current  baseline  estimate  for  the  program  or 

•  an  APUC  that  is  least  25  percent  over  the  acquisition  procurement  unit  cost 
shown  in  the  current  baseline  estimate  for  the  program  or 

•  a  PAUC  that  is  at  least  50  percent  over  the  program  acquisition  unit  cost  shown 
in  the  original  baseline  estimate  or 

•  an  APUC  that  is  at  least  50  percent  over  the  acquisition  procurement  unit  cost 
shown  in  the  original  baseline  estimate. 

Table  A.l  summarizes  the  unit  cost  threshold  definitions  in  federal  law. 


Table  A.l 

Breach  Thresholds 


Unit 

Cost 

Baseline 

Threshold, 

% 

Level 

Source 

PAUC 

Current 

>15 

Significant 

10  use  §  2433  (a)(4)(A)(i) 

APUC 

Current 

>15 

Significant 

10  USC  §  2433  (a)(4)(B)(i) 

PAUC 

Original 

>30 

Significant 

10  USC  §  2433  (a)(4)(A)(ii) 

APUC 

Original 

>30 

Significant 

10  USC  §  2433  (a)(4)(B)(ii) 

PAUC 

Current 

>25 

Critical 

10  USC  §  2433  (a)(5)(A)(i) 

APUC 

Current 

>25 

Critical 

10  USC  §  2433  (a)(5)(B)(i) 

PAUC 

Original 

>50 

Critical 

10  USC  §  2433  (a)(5)(A)(ii) 

APUC 

Original 

>50 

Critical 

10  USC  §  2433  (a)(5)(B)(ii) 
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Certifications  to  Congress 

Current  law  specifies  that  a  program  that  incurs  a  critical  cost  growth  breach  is  assumed 
to  be  terminated  unless  the  Secretary  of  Defense  provides  specific  certifications  to 
Congress  within  60  days  of  the  SAR  due  date  as  specified  in  10  USC  §  2432(f).  The 
certifications  must  state  the  following: 

•  The  system  is  essential  to  national  security. 

•  There  are  no  alternatives  that  will  provide  acceptable  capability  to  meet  joint  mili¬ 
tary  requirements  at  less  cost. 

•  The  new  estimates  of  PAUC  and  APUC  have  been  determined  by  the  defense 
director  of  Cost  Analysis  and  Program  Evaluation  to  be  reasonable. 

•  The  program  is  a  higher  priority  than  programs  whose  funding  must  be  reduced 
to  accommodate  the  growth  in  cost  of  the  program. 

•  The  management  structure  for  the  program  is  adequate  to  manage  and  control 
the  PAUC  or  APUC  going  forward. 

These  certifications  must  be  accompanied  by  a  root  cause  analysis  and  assessment 
(described  in  following  section). 

•  10  USC  §  2433a  stipulates  that  a  root  cause  analysis  be  conducted  by  the  director 
of  PARCA  when  a  major  weapon  system  acquisition  program  incurs  a  critical  cost 
growth  breach.  The  root  cause  analysis  assesses  the  underlying  cause  of  the  critical 
cost  overrun.  As  described  in  Chapter  One,  the  office  is  responsible  for  producing 
a  root  cause  report  that  must  identify  the  role,  if  any,  of  the  eight  items  stipulated. 


Root  Cause  Analysis 

10  USC  §  2433a  stipulates  that  a  root  cause  analysis  be  conducted  by  the  director  of 
PARCA  when  a  major  weapon  system  acquisition  program  incurs  a  critical  cost  growth 
breach.  The  root  cause  analysis  assesses  the  underlying  cause  of  the  critical  cost  over¬ 
run.  The  root  cause  report  must  identify  the  role,  if  any,  of  the  following  elements  in 
causing  the  cost  overrun: 

•  unrealistic  performance  expectations 

•  unrealistic  baseline  estimates  for  cost  or  schedule 

•  immature  technologies  or  excessive  manufacturing  or  integration  risk 

•  unanticipated  design,  engineering,  manufacturing,  or  technology  integration 
issues  arising  during  program  performance 

•  changes  in  procurement  quantities 

•  inadequate  program  funding  or  funding  instability 
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•  poor  performance  by  government  or  contractor  personnel  responsible  for  pro¬ 
gram  management 

•  any  other  matters. 


APPENDIX  B 


Vulnerability  Matrix 


Independent  of  program,  program  analysis  and  root  cause  analysis  of  cost  and  schedule 
overruns  may  show  common  themes.  RAND  studied  four  programs  during  the  root 
cause  analysis  project:  Joint  Strike  Fighter,  DDG-1000,  Wideband  Global  Satellites, 
and  the  Longbow  Apache  Helicopter.  Although  each  program  breached  the  Nunn- 
McCurdy  Act  for  different  reasons,  it  is  useful  to  look  for  any  common  themes  in  the 
breach  and  common  themes  in  overall  risk  that  could  help  lawmakers  identify  poten¬ 
tial  breaches  earlier. 

Table  B.l  shows  a  compiled  list  of  potential  risks  and  events  that  could  lead  to 
a  Nunn-McCurdy  breach,  along  with  the  four  programs  that  were  studied  as  part  of 

Table  B.l 

Vulnerability  Matrix  with  Program  Findings 


DDG- 

Vulnerability  Matrix:  Type  of  Event  JSF  1000  WGS  Apache 

Cost  increases — concept  refinement  x 

Cost  increases — technology  development  x 

Cost  increases — systems  development  and  demonstration  x  x  x 

Cost  increases — production  and  deployment 

Cost  increases — operations  and  support 

Cost  increases — additional  blocks  x 

Documented  different  estimates — concept  refinement  x  x 

Documented  different  estimates — technology  development  x  x 

Documented  different  estimates — systems  development  and  x  x  x 

demonstration 

Documented  different  estimates — production  and  deployment  x 

Documented  different  estimates — operations  and  support 

Documented  different  estimates — additional  blocks  x 
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Table  B.1 — Continued 


Vulnerability  Matrix:  Type  of  Event 

JSF 

DDG- 

1000 

WGS  Apache 

Known  Factors  During  Cost  Estimation 

Unrefined  design  and  performance  requirements 

X 

X 

Lack  of  previous  similar  work 

X 

X 

X 

No  or  bad  past  contractor  performance 

X 

Technology  immaturity 

X 

X 

Technical  complexity 

X 

X 

Design  flaws 

X 

New  manufacturing  methods 

X 

X 

Integration  of  system  is  complex 

X 

X 

Decision  to  employ  parallel  development  rather  than  sequential 

X 

X 

Insufficient  number  of  government  personnel  to  run  the  program 

X 

Unique  contracting  strategy 

X 

Ambitious  system  performance  goals 

X 

X 

International  participation 

X 

Difficulties  in  negotiating  major  contracts  (e.g.,  contract  protests) 

X 

Internal  Changes  to  Program 

Change  of  performance  requirements — concept  refinement 

X 

Change  of  performance  requirements — technology  development 

X 

Change  of  performance  requirements — systems  development  and 
demonstration 

X 

Change  of  performance  requirements — production  and  deployment 

Change  of  performance  requirements — operations  and  support 

Change  of  performance  requirements — additional  blocks 

Funding  instability 

X 

Funding  insufficiency 

Optimistic  development  schedule 

X 

Schedule  delays  and  extension 

X 

Change  in  quantity — increase 

X 

X  X 

Change  in  quantity — decrease 

X 

X 
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Table  B.1 — Continued 


Vulnerability  Matrix:  Type  of  Event 

JSF 

DDG- 

1000 

WGS 

Apache 

External  Changes  to  Program  Environment 

Increase/decrease  in  quantity  of  related  program 

X 

X 

X 

Cancellation  of  predecessor  program 

X 

X 

Program  restructured 

Reliance  on  private  sector  to  maintain  technology  configuration 

X 

X 

Changes  in  program  guidance/service  mission 

X 

Changes  to  external  programs 

X 

Addition  of  new  construction  to  an  upgrade  program 

X 

"Special  interest"  program 

Change  of  technology  direction 

Changes  to  Operating  Environment 

Increase  in  wage  rates 

X 

X 

X 

Increase  in  cost  of  materials  or  equipment 

X 

X 

X 

X 

Changes  to  industrial  base 

X 

X 

Program  decisions  heavily  influenced  by  industrial  base  concerns 

Change  in  government  standards  or  regulations 

Natural  disaster/act  of  God 

X 

Strong  advocacy  for  program  in  OSD 

X 

X 

Strong  advocacy  against  program  in  OSD 

X 

X 

Strong  advocacy  for  program  in  Services 

X 

X 

Strong  advocacy  against  program  in  Services 

X 

X 

Strong  advocacy  for  program  in  Congress 

X 

X 

Strong  advocacy  against  program  in  Congress 

X 

X 

Significant  congressional  interest 

X 

X 

this  effort.  It  is  important  to  note  that  this  matrix  does  not  only  represent  the  reasons 
for  the  Nunn-McCurdy  breach  of  the  four  programs.  It  also  represents  a  broader  look 
at  the  risks  that  programs  could  potentially  encounter.  Better  understanding  of  factors 
that  programs  can  encounter  as  they  head  toward  a  Nunn-McCurdy  breach  could  help 
lawmakers  and  Program  Offices  more  quickly  identify  at-risk  programs  so  that  adjust¬ 
ments  could  be  made. 
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Table  B.2  compares  the  vulnerability  matrix  (VM)  to  two  other  lists  of  warning 
signs.  First  is  the  DoD  probability  of  program  success  (PoPS)  factors.  Second  is  a  list 


Table  B.2 

Vulnerability  Matrix  Mapped  to  PoPS  and  Young's  Memo  Findings 


Vulnerabilty  Matrix:  Type  of  Event 

Mapping 

Adequacy 

Generic  PoPS 

Young's  Memo 

Cost  increases — concept  refinement 

Y 

Program 

execution: 

Cost/schedule 

performance 

Development 

issues 

Cost  increases — technology 
development 

Y 

Program 

execution: 

Cost/schedule 

performance 

Development 

issues 

Cost  increases — systems  development 
and  demonstration 

Y 

Program 

execution: 

Cost/schedule 

performance 

Development 

issues 

Cost  increases — production  and 
deployment 

Y 

Program 

execution: 

Cost/schedule 

performance 

Development 

issues 

Cost  increases — operations  and  support 

Y 

Program 

execution: 

Sustainability  risk 

Development 

issues 

Cost  increases — additional  blocks 

Y 

Program 

execution: 

Cost/schedule 

performance 

Development 

issues 

Documented  different  estimates — 
concept  refinement 

Y 

Program 

execution: 

Cost  estimating 

Low  estimates 

Documented  different  estimates — 
technology  development 

Y 

Program 

execution: 

Cost  estimating 

Low  estimates 

Documented  different  estimates — 
systems  development  and 
demonstration 

Y 

Program 

execution: 

Cost  estimating 

Low  estimates 

Documented  different  estimates — 
production  and  deployment 

Y 

Program 

execution: 

Cost  estimating 

Low  estimates 

Documented  different  estimates — 
operations  and  support 

Y 

Program 

execution: 

Cost  estimating 

Low  estimates 

Documented  different  estimates — 
additional  blocks 

Y 

Program 

execution: 

Cost  estimating 

Low  estimates 

Known  Factors  During  Cost  Estimation 

Unrefined  design  and  performance 
requirements 

Y 

Program 

requirements: 

Program  scope 
evolution 

Flawed  design 
process  and 
knowledge 
base 

Lack  of  previous  similar  work 

O 

Program 

execution: 

Technical 

maturity 

No  or  bad  past  contractor  performance 

Y 

Program 

execution: 

Contractor 

performance 
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Table  B.2 — Continued 

Vulnerabilty  Matrix:  Type  of  Event 

Mapping 

Adequacy 

Generic  PoPS 

Young's  Memo 

Technology  immaturity 

Y 

Program 

execution: 

Technical 

maturity 

Inadequate 

prototypes, 

immature 

technology, 

too  little 

understanding 

of  the 

design,  risky 
technology 

Technical  complexity 

Y 

Program 

execution: 

Technical 

maturity 

Exquisite 

technology 

Design  flaws 

Y 

Program 

execution: 

Technical 

maturity 

Design  flaws 

New  manufacturing  methods 

Y 

Invalid 

remanufacture 

assumptions 

Integration  of  system  is  complex 

0 

Program 

execution: 

Program  risk 
assessment 

Decision  to  employ  parallel 
development  rather  than  sequential 

R 

Insufficient  number  of  government 
personnel  to  run  the  program 

G 

Program 

resources: 

Manning 

Unique  contracting  strategy 

R 

Ambitious  system  performance  goals 

Y 

Program 

execution: 

Software 

International  participation 

G 

Program 

advocacy: 

International 

Difficulties  in  negotiating  major 
contracts  (e.g.,  contract  protests) 

0 

Program 

requirements: 

Program  scope 
evolution 

Internal  Changes  to  Program 


Change  of  performance  requirements — 

Y 

Program 

Program  scope 

Changing 

concept  refinement 

requirements: 

evolution 

requirements, 
new  systems 

Change  of  performance  requirements — 

Y 

Program 

Program  scope 

and  capabilities 

technology  development 

requirements: 

evolution 

added, 

excessive 

Change  of  performance  requirements — 
systems  development  and 
demonstration 

Y 

Program 

requirements: 

Program  scope 
evolution 

requirements 

Change  of  performance  requirements — 
production  and  deployment 

Y 

Program 

requirements: 

Program  scope 
evolution 

Change  of  performance  requirements — 
operations  and  support 

Y 

Program 

execution: 

Sustainability  risk 
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Table  B.2 — Continued 


Vulnerabilty  Matrix:  Type  of  Event 

Mapping 

Adequacy 

Generic  PoPS 

Young's  Memo 

Change  of  performance  requirements — 
additional  blocks 

Y 

Program 

requirements: 

Program  scope 
evolution 

Funding  instability 

Y 

Program 

resources: 

Budget 

Fluctuating 

budgets 

Funding  insufficiency 

G 

Program 

resources: 

Budget 

Underfunding 

Optimistic  development  schedule 

Y 

Program 

execution: 

Cost/schedule 

performance 

Optimistic 

schedule 

Schedule  delays  and  extension 

Y 

Program 

execution: 

Cost/schedule 

performance 

Extended 

schedule 

Change  in  quantity — increase 

R 

Quantity 

increased 

Change  in  quantity — decrease 

R 

Reduced 
annual 
quantities, 
precipitous 
drop  in 
quantity, 
reduced  annual 
production  rate 

External  Changes  to  Program  Environment 

Increase/decrease  in  quantity  of  related 
program 

0 

Program  "fit" 
in  capability 
vision 

DoD  vision — inter 
operability 

Cancellation  of  predecessor  program 

0 

Program  "fit" 
in  capability 
vision 

DoD  vision — inter 
operability 

Program  restructured 

R 

Reliance  on  private  sector  to  maintain 
technology  configuration 

R 

Changes  in  program  guidance/service 
mission 

Y 

Program  "fit" 
in  capability 
vision 

Service  vision — 
current  force 

Fluid  program 
strategy 

Changes  to  external  programs 

R 

Addition  of  new  construction  to  an 
upgrade  program 

R 

"Special  interest"  program 

R 

Change  of  technology  direction 

R 

Increase  in  wage  rates 


Changes  to  Operating  Environment 


R 
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Table  B.2 — Continued 


Mapping 

Vulnerabilty  Matrix:  Type  of  Event  Adequacy  Generic  PoPS 


Young's  Memo 


Increase  in  cost  of  materials  or 
equipment 


Changes  to  industrial  base 

Program  decisions  heavily  influenced  by 
industrial  base  concerns 

Change  in  government  standards  or 
regulations 

Natural  disaster/act  of  God 
Strong  advocacy  for  program  in  OSD 

Strong  advocay  against  program  in  OSD 

Strong  advocacy  for  program  in  Services 


Strong  advocacy  against  program  in 
Services 

Strong  advocacy  for  program  in 
Congress 

Strong  advocay  against  program  in 
Congress 

Significant  congressional  interest 


Program 

execution: 

Industry/company 

assessment 

Program 

execution: 

Industry/company 

assessment 

Program 

advocacy: 

OSD 

Program 

advocacy: 

OSD 

Program 

advocacy: 

Service  secretary 

Program 

advocacy: 

Service  secretary 

Program 

advocacy: 

Congress 

Program 

advocacy: 

Congress 

Program 

advocacy: 

Congress 

Congress 
reversed  DoD 
decision 

of  items  discussed  in  the  January  30,  2009,  memo  from  Under  Secretary  of  Defense 
John  Young  (OUSD  AT&L)  to  Secretary  of  Defense  Robert  Gates  regarding  his  own 
reasoning  for  cost  changes  in  large  acquisition  programs. 

Table  B.3  shows  what  additional  data  and  manpower  might  be  needed  if  all  the 
VM  factors  were  to  be  recorded  and  acted  on. 


Ability  of  DoD  to  Identify  Programs  at  Risk 

Common  causal  factors  emerged  during  the  Nunn-McCurdy  root  cause  analysis.  These 
factors  might  help  identify  additional  programs  at  risk  for  cost  or  schedule  problems. 
Yet,  it  appears  PARCA  might  not  receive  this  information  in  easily  recognizable  for¬ 
mats.  RAND  identified  metrics  that  might  highlight  critical  factors  in  potentially  vul- 
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Table  B.3 

Increased  Manpower  and  Reporting  Needed  to  Adequately  Report  VM  Items 


Vulnerability  Matrix:  Type  of 

Event 

Increased 

Manpower 

Additional 
Reports  from 
Program  Office 

Additional 
Reports  from 
Contractor 

Should 
Already  Be 
Recorded 

Cost  increases — concept 
refinement 

X 

Cost  increases — technology 
development 

X 

Cost  increases — systems 
development  and  demonstration 

X 

Cost  increases — production  and 
deployment 

X 

Cost  increases — operations  and 
support 

X 

Cost  increases — additional  blocks 

X 

Documented  different 
estimates — concept  refinement 

X 

Documented  different 
estimates — technology 
development 

X 

Documented  different 
estimates — systems  development 
and  demonstration 

X 

Documented  different 
estimates — production  and 
deployment 

X 

Documented  different 
estimates — operations  and 
support 

X 

Documented  different 
estimates — additional  blocks 

X 

Known  Factors  During  Cost  Estimation 

Unrefined  design  and 
performance  requirements 

X 

Lack  of  previous  similar  work 

X 

X 

No  or  bad  past  contractor 

X 

X 

performance 

Technology  immaturity  x 

Technical  complexity  x 

Design  flaws 

New  manufacturing  methods 


x 
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Table  B.3 — Continued 


Additional 

Additional 

Should 

Vulnerability  Matrix:  Type  of 

Increased 

Reports  from 

Reports  from 

Already  Be 

Event 

Manpower 

Program  Office 

Contractor 

Recorded 

Complex  integration  of  system 

X 

Decision  to  employ  parallel 
development  rather  than 
sequential 

X 

Insufficient  number  of 
government  personnel  to  run  the 
program 

X 

Unique  contracting  strategy 

X 

Ambitious  system  performance 
goals 

X 

International  participation 

X 

Difficulties  in  negotiating  major 
contracts  (e.g.,  contract  protests) 

X 

Internal  Changes  to  Program 

Change  of  performance 
requirements — concept 
refinement 

X 

X 

Change  of  performance 
requirements — technology 
development 

X 

X 

Change  of  performance 
requirements — systems 
development  and  demonstration 

X 

X 

Change  of  performance 
requirements — production  and 
deployment 

X 

X 

Change  of  performance 
requirements — operations  and 
support 

X 

X 

Change  of  performance 
requirements — additional  blocks 

X 

X 

Funding  instability 

X 

Funding  insufficiency 

X 

Optimistic  development  schedule 

Schedule  delays  and  extension 

Change  in  quantity — increase 

X 

Change  in  quantity — decrease 

X 
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Table  B.3 — Continued 


Vulnerability  Matrix:  Type  of 
Event 

Additional 

Increased  Reports  from 

Manpower  Program  Office 

Additional 
Reports  from 
Contractor 

Should 
Already  Be 
Recorded 

External  Changes  to  Program 

Increase/decrease  in  quantity  of 
related  program 

X 

Cancellation  of  predecessor 
program 

X 

Program  restructured 

X 

Reliance  on  private  sector 
to  maintain  technology 
configuration 

X 

Changes  in  program  guidance/ 
service  mission 

X 

Changes  to  external  programs 

X 

Addition  of  new  construction  to 
an  upgrade  program 

X 

"Special  interest"  program 

Change  of  technology  direction 

X 

X 

Changes  to  Operating  Environment 

Increase  in  wage  rates 

X 

Increase  in  cost  of  materials  or 
equipment 

X 

Changes  to  industrial  base 

X 

Program  decisions  heavily 
influenced  by  industrial  base 
concerns 

X 

Change  in  government  standards 
or  regulations 

X 

Natural  disaster/act  of  God 

X 

Strong  advocacy  for  program  in 
OSD 

Strong  advocacy  against  program 
in  OSD 

Strong  advocacy  for  program  in 
Services 


Strong  advocacy  against  program 
in  Services 
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Table  B.3 — Continued 


Vulnerability  Matrix:  Type  of 

Event 

Increased 

Manpower 

Additional 
Reports  from 
Program  Office 

Additional 
Reports  from 
Contractor 

Should 
Already  Be 
Recorded 

Strong  advocacy  for  program  in 
Congress 

Strong  advocay  against  program 
in  Congress 

Significant  congressional  interest 

X 

nerable  programs.  These  metrics  were  compared  to  the  PoPS  criteria  possibly  received 
by  PARCA.  RAND  identified  those  metrics  PoPS  might  not  identify  but  should  given 
previous  root  cause  results. 

All  three  services  use  the  PoPS  metrics. 

•  Army:  “The  probability  of  program  success  initiative  is  designed  to  improve  the 
Army’s  ability  to  accurately  assess  a  program’s  probability  of  success,  and  clearly/ 
concisely  represent  that  success  probability  to  Army  leadership.  The  probability  of 
program  success  will  be  calculated  monthly.  The  report  to  senior  leadership  will 
occur  quarterly  during  the  following  months:  January,  April,  July,  and  October. 
The  senior  leadership  report  can  be  generated  more  frequently,  if  required.”1 

•  Navy:  PoPS  provides  a  “holistic  view  of  overall  program  health  and  readiness  to 
proceed.”2 

•  Air  Force:  “The  probability  of  program  success  initiative  is  designed  to  improve 
the  Air  Force’s  ability  to  accurately  assess  a  program’s  probability  of  success  (abil¬ 
ity  to  succeed),  and  clearly/concisely  represent  that  success  probability  to  Air  Force 
leadership.  To  that  end,  each  program  spiral  (including  software  spirals)  will  be 
evaluated  independently  utilizing  the  probability  of  program  success  criteria.”3 

PoPS  gives  the  Program  Office  the  ability  to  display  a  large  amount  of  data  in  a 
standardized  format  to  facilitate  senior  leadership  decisionmaking.  The  standardized 
format  allows  for  quick  comparisons  across  programs  and  for  historical  analysis  of  pro¬ 
gram  performance. 

However,  PoPS  does  not  quickly  identify  aspects  of  change  throughout  the  life- 
cycle  of  a  program.  For  example,  PoPS  does  not  highlight  change  in  the  quantity  of 
a  related  program,  which  we  found  to  be  a  major  factor  in  Nunn-McCurdy  breaches. 


1  U.S.  Army,  Probability  of  Program  Success  Operations  Guide. 

2  U.S.  Navy,  Naval  Probability  of  Program  Success  (PoPS)  Executive  Brief  June  2009. 

3  U.S.  Air  Force,  Probability  of  Program  Success  (PoPS)  Spreadsheet  Operations  Guide,  July  2007. 
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PoPS  represents  a  snapshot  of  a  program  as  it  exists  at  a  point  in  time,  which  limits  its 
utility  for  predicting  future  breaches.  Table  B.2  ranks  the  mapping  of  PoPS  to  the  VM 
from  red  to  green,  with  green  being  a  good  match  and  red  meaning  that  PoPS  does 
not  identify  the  issue  at  all.  The  following  list  shows  what  PoPS  does  well  and  what  it 
does  not  do  well. 

PoPS  Covers  Some  Potential  Root  Cause  Factors 

•  internal  factors 

-  sufficient  government  personnel  to  run  a  program 

-  funding  sufficiency 

•  external  factors 

-  international  advocacy  and  participation 

-  program  advocacy  for  a  program  from  OSD,  services,  and  Congress. 

Some  Criteria  in  PoPS  Can  Be  Altered  Slightly  to  Highlight  the  Impact  of  These 
Critical  Factors 

•  internal  factors  (program  incongruency) 

-  different  cost  estimates  as  a  program  progresses 

-  changes  in  performance  requirements 

-  funding  instability  (different  from  funding  sufficiency) 

•  internal  changes  (program  ambition  understanding) 

-  ambitious  system  engineering  goals 

-  technical  immaturity  or  complexity 

-  cost  increases  as  program  progresses 

•  external  factors 

-  congressional  interest  (different  from  advocacy) 

•  external  changes 

-  new  manufacturing  techniques 

-  changes  to  the  industrial  base. 

PoPS  Only  Tangentially  Addresses  the  Impact  of  Some  Critical  Factors 

•  internal  factors  (program  process  decisions) 

-  source  selection  decisions: 

-  lack  of  previous  similar  work  at  a  contractor 

-  integration  of  a  complex  system,  even  if  system  is  made  up  of  noncomplex 
parts 

-  contract  negotiation  concessions 
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•  external  changes 

-  increase  or  decrease  in  related  program  quantity 

-  cancellation  of  a  predecessor  program. 

PoPS  Is  Missing  Potentially  Critical  Parameters 

•  internal  changes 

-  changes  in  quantity:  decreases  or  increases 

-  changes  to  material,  equipment,  or  manpower  costs 

-  restructuring  of  a  program 

•  impact  of  internal  factors  (program  process  decisions) 

-  parallel  development  vs.  sequential  development 

•  external  changes 

-  changes  to  external  programs 

-  changes  to  commercial  sector  for  which  program  dependent  to  maintain  a 
technology 

-  changes  in  government  standards  or  regulations 

•  external  factors 

-  special  interest  programs  involvement 

-  natural  disasters/acts  of  God. 

PoPS  contains  some  criteria  that  can  be  helpful  to  identifying  potential  cost  and 
schedule  overruns  that  lead  to  Nunn-McCurdy  breaches.  However,  most  criteria  need 
to  be  adjusted  to  better  highlight  potential  root  causes.  Some  RCA  indicators  identified 
by  RAND  do  not  exist  in  PoPs  criteria.  PoPS  does  not  account  for  the  changes  in  cost 
or  schedule,  just  gives  a  snapshot  in  time.  Therefore,  those  who  look  only  at  PoPS  do 
not  see  changes  that  have  occurred  in  the  program. 


Programs  Are  in  Danger  for  Similar  Reasons 

In  the  course  of  the  study,  we  found  that  all  four  programs  sometimes  shared  risk  fac¬ 
tors  that  lead  to  cost  increases  and  schedule  slips.  Table  B.l  shows  the  types  of  events 
that  each  program  experienced.  Forty-five  percent  of  the  vulnerability  matrix  factors 
were  experienced  by  more  than  one  program.  Some  important  factors  that  affected 
three  or  more  programs  are  those  listed  below: 

•  lack  of  previous  similar  work 

•  increased  cost  of  material 

•  increase  in  wage  rates 


112  Root  Cause  Analyses  of  Nunn-McCurdy  Breaches,  Volume  1 


•  increase  in  quantity 

•  cost  increases  and  documented  cost  estimate  differences  during  system  develop¬ 
ment  and  demonstration. 

Other  notable  factors  include: 

•  reliance  on  the  private  sector  to  maintain  technology  configuration 

•  strong  advocacy  for  or  against  programs 

•  complex  system  integration 

•  insufficient  number  of  government  personnel  on  the  program. 

By  studying  the  vulnerability  matrix,  we  can  see  that  unique  programs  share  the 
same  warning  signs  that  can  lead  to  a  Nunn-McCurdy  breach.  (See  Tables  B.2  and 
B.3.) 


APPENDIX  C 

Technology  Readiness  Levels 


Table  C.l  presents  the  definitions  of  the  technology  readiness  levels  used  in  DoD 
acquisitions. 

Table  C.l 

Definitions  of  DoD  Technology  Readiness  Levels 


Technology  Readiness  Level 

Description 

1.  Basic  principles  observed  and 
reported 

Lowest  level  of  technology  readiness.  Where  scientific 
research  begins  to  be  later  translated  into  applied  research 
and  development.  Examples  might  include  paper  studies  of  a 
technology's  basic  properties. 

2.  Technology  concept  or  application 
formulated 

Invention  begins.  Once  basic  principles  are  observed,  practical 
applications  can  be  invented.  The  application  is  speculative  and 
there  is  no  proof  or  detailed  analysis  to  support  the  assumption. 
Examples  are  still  limited  to  paper  studies. 

3.  Analytical  and  experimental 
critical  function  or  characteristic 
proof  of  concept 

Active  research  and  development  is  initiated.  This  includes 
analytical  studies  and  laboratory  studies  to  physically  validate 
analytical  predictions  of  separate  elements  of  the  technology. 
Examples  include  components  that  are  not  yet  integrated  or 
representative. 

4.  Component  or  breadboard 
validation  in  laboratory  environment 

Basic  technological  components  are  integrated  to  establish  that 
the  pieces  will  work  together.  This  is  relatively  "low  fidelity" 
compared  to  the  eventual  system.  Examples  include  integration 
of  ad  hoc  hardware  in  a  laboratory. 

5.  Component  or  breadboard 
validation  in  relevant  environment 

Fidelity  of  breadboard  technology  increases  significantly.  The 
basic  technological  components  are  integrated  with  reasonably 
realistic  supporting  elements  so  that  the  technology  can  be 
tested  in  a  simulated  environment.  Examples  include  "high- 
fidelity"  laboratory  integration  of  components. 

6.  System/subsystem  model  or 
prototype  demonstration  in  a 
relevant  environment 

Representative  model  or  prototype  system,  which  is  well 
beyond  the  breadboard  tested  for  TRL  5,  is  tested  in  a  relevant 
environment.  Represents  a  major  step  up  in  a  technology's 
demonstrated  readiness.  Examples  include  testing  a  prototype 
in  a  high-fidelity  laboratory  environment  or  in  a  simulated 
operational  environment. 

7.  System  prototype  demonstration 
in  an  operational  environment 

Prototype  near  or  at  planned  operational  system.  Represents 
a  major  step  up  from  TRL  6,  requiring  the  demonstration  of  an 
actual  system  prototype  in  an  operational  environment,  such  as 
in  an  aircraft,  a  vehicle,  or  space.  Examples  include  testing  the 
prototype  in  a  test  bed  aircraft. 
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Table  C.1 — Continued 


Technology  Readiness  Level 

Description 

8.  Actual  system  completed  and 
flight  qualified  through  test  and 
demonstration 

Technology  has  been  proven  to  work  in  its  final  form  and  under 
expected  conditions.  In  almost  all  cases,  this  TRL  represents 
the  end  of  true  system  development.  Examples  include 
developmental  test  and  evaluation  of  the  system  in  its  intended 
weapon  system  to  determine  if  it  meets  design  specifications. 

9.  Actual  system  flight  proven  Actual  application  of  the  technology  in  its  final  form  and  under 

through  successful  mission  operations  mission  conditions,  such  as  those  encountered  in  operational 

test  and  evaluation.  In  almost  all  cases,  this  is  the  end  of  the 
last  "bug  fixing"  aspects  of  true  system  development.  Examples 
include  using  the  system  under  operational  mission  conditions. 


SOURCE:  Department  of  Defense,  DODI  5000.2  Acquisition  System  Guidebook,  2004. 


APPENDIX  D 


Cost  Estimates  from  Other  Organizations 


Chapter  Four  showed  the  cost  estimates  in  the  SAR.  This  appendix  shows  the  cost  esti¬ 
mates  from  Lockheed  Martin,  JPO,  and  the  Air  Force  Cost  Analysis  Agency. 


Lockheed  Martin 

Lockheed  Martin  provided  a  chart  explaining  URF  cost  growth  in  the  RDT&E 
phase.  Extrapolating  from  the  chart,  it  appears  that  Lockheed  attributes  approximately 
$20  billion  (roughly  one-third)  of  RDT&E  cost  growth  to  the  weight  and  design  issues 
that  were  identified  and  addressed  between  2001  and  2004.  Other  factors  contributed 
to  another  $20  billion  in  the  air  system  contract  with  another  $20  billion  occurring  in 
the  propulsion  systems  contract.1 


JPO 

The  JPO  provided  RAND  with  an  estimate  of  the  factors  contributing  to  APUC, 
which  is  the  procurement  account  only.  Table  D.l  shows  the  JPO  assessment.  The  JPO 
allocates  almost  30  percent  of  cost  growth  to  three  factors:  design/weight  issues,  cost  of 
materials,  and  labor  rates.  Another  7.4  percent  is  allocated  to  propulsion  system  costs. 
Finally,  the  JPO  allocates  6.3  percent  of  the  cost  growth  to  OSD  inflation  changes. 


CAIG/CAPE 

A  fourth  set  of  data  comes  from  the  CAIG,  now  CAPE,  which  redid  its  estimate  in 
2004-2005  in  the  wake  of  the  re-baselining  of  the  program  after  the  weight  and  design 
issues  arose.  However,  these  numbers,  which  describe  URF  cost  growth,  were  not  used 
as  the  program  estimate  until  2008,  when  the  update  was  calculated  through  2015. 


1  Authors’  interpretation  of  Lockheed  Martin  chart  provided  April  23,  2010. 
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Table  D.1 

JPO  Track  of  Average  Procurement  Unit  Cost 
Growth  (APUC  Only,  MS  B  to  SAR  2009) 


Cost  Growth  Factor 

Percentage 

Design  change/weight 

11.0 

Cost  of  materials 

9.0 

Labor  rates 

8.5 

Propulsion  update 

7.4 

OSD  inflation  changes 

6.3 

Aircraft  reduction  and  multiyear  shifts 

4.5 

Production  support  update 

3.1 

Airframe  update 

2.8 

Rate  tooling 

2.0 

CAPE  estimate 

1.3 

Other 

0.9 

Systems  update 

0.7 

Total 

57.5 

SOURCE:  Data  provided  by  JPO  to  PARCA  JSF  teams 
meeting,  April  23,  2010. 


The  CAIG  estimate  is  yet  another  breakout  of  the  program,  providing  some  overlap 
with  categories  reported  by  Lockheed  Martin  and  the  JPO,  but  it  is  not  a  clean  match. 
These  data  are  portrayed  in  Table  D.2. 
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Table  D.2 

CAPE  Track  of  CTOL  Unit  Recurring  Flyaway  Cost  Growth 
(FY  2002  $  Millions) 


Cost 

Percentage 

Change 

Milestone  B  estimate 

36.3 

Updated  avionics  from  F-22  actuals 

5.5 

15 

Weight  growth 

5.1 

14 

Updated  labor  rates 

3.8 

10 

Fee  and  fee-on-fee 

3.5 

10 

Updated  propulsion  from  F-22/F1 19  actuals 

2.5 

7 

Quantity  reduction 

1.0 

3 

Change  in  estimate  since  Milestone  B 

(21.3) 

59 

Updated  URF  estimate 

57.6 

SOURCE:  "URF  Cost  Comparisons  from  Milestone  B,"  draft/pre- 
decisional  briefing  slide  provided  by  CAPE  to  PARCA  team  on  April  28, 
2010. 


APPENDIX  E 

Effect  of  Low  Original  WGS  Baseline  Costs 


The  SAF  letter  explaining  the  WGS  Nunn-McCurdy  breach  argued  that  Boeing’s  fail¬ 
ure  to  price  Block  I  satellites  correctly  meant  that  the  original  and  the  current  APB 
were  below  the  true  cost  of  building  satellites  (thus,  a  reversion  to  correct  costs  appears 
to  be  a  cost  increase).  It  is  not  clear  what  Block  I  satellites  should  have  cost,  since  Boe¬ 
ing’s  FFP  contract  did  not  require  that  it  submit  cost  data  at  the  time,  and  recently 
released  cost  data  had  yet  to  be  thoroughly  audited  at  the  time  of  this  writing.  The  cost 
claims  currently  made  by  Boeing  suggest  that  the  true  cost  of  the  first  three  satellites 
was  roughly  the  same  as  the  next  two  satellites,  despite  higher-than-normal  inflation  in 
the  satellite  industry  and  the  extra  features  present  in  Block  II,  which  is  both  unusual 
and  counterintuitive. 

Were  such  a  claim  nevertheless  true,  then  the  unit  cost  for  satellites  1-3  would 
have  been  $377  million  rather  than  $239  million,  and  APUC  for  the  current  APB 
would  be  $377  million  (rather  than  $294  million).  In  comparison,  the  projected 
APUC  increase  would  have  been  15  percent  (five  satellites  at  $377  million  and  two  at 
$574  million  for  a  total  of  $2,998  million  for  seven  satellites  or  an  APUC  of  $428  mil¬ 
lion).  The  $428  million  APUC  is  14.5  percent  higher  than  the  $377  million  projection. 
It  still  qualifies  as  a  significant  cost  growth  threshold  breach,  albeit  borderline.  In  light 
of  USAF  plans  to  buy  WGS  9  through  12,  however,  a  critical  cost  growth  threshold 
breach  would  occur  by  WGS  11  as  illustrated  by  the  red-shaded  cells  of  Table  E.l  (if 
the  last  four  satellites  in  the  Block  II  follow-on  series  had  the  same  unit  cost  as  the  first 
two).  In  other  words,  regardless  of  whether  or  not  Block  I  prices  reflected  actual  costs, 
the  52  percent  increase  in  unit  costs  warrants  explanation. 

Table  E.l 

WGS  Eventual  Nunn-McCurdy  Breach 

Satellite  8  9  10  11  12 

APUC,  $  millions  433  451  464  475  484 

Increase  over  APB,  % _ 15 _ 20 _ 23  j 

NOTE:  Assuming  that  WGS  APUC  for  Blocks  I  and  II  should  be 

$377  million. 
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